Conclusion: most people may not notice the rolling shutter in the background in a portrait-mode horizontal pan SOOC, but there is always rolling shutter if the shutter rolls and the lens moves
If your subject went left to right and you panned it, and then it went right to left, and you panned it again, showing those images in sequence would reveal their difference in horizontal compression.