One thing I've noticed is anytime people of another ethnicity speak in their own language and laugh or smile or just seem to have a good time, some white person thinks they are talking about them. I experienced this so much working retail where our employees would have a conversation and customers would accuse them of talking behind their back.
The bizarre assumption everything is about you is super tiring.
I think this is one of those things that is very specific to people who don't travel much and have lived their whole lives in one culture/location. When people don't understand what others are saying, depending on their temperament and background, this can be a very unsettling thing. And if they're even remotely insecure, not knowing what other people are saying can generate huge amounts of paranoia.
But the second that you travel--or better yet live in--another country, you have to drop that mindset pretty fast. I spent over 10 years living in Singapore, where the three primary cultures are Chinese, Malaysian and Indian, and they all spoke among their own friends and family in their ancestral language while communicating across cultures in English. You get used to it pretty fast, hearing multiple different languages you don't understand, and not automatically assuming EVERYONE *must* be talking about you, because you don't know what they're saying. Paranoia can't sustain that level of insecurity for too long, I imagine eventually you just get tired of being suspicious of everyone around you.