Yeah, this is right. A lot of people come to the UK to work, sometimes temporarily. I've never heard the term expat being used here, however... which does make it suspect to me. I have indeed only heard it being used for westerners, particularly white westerners, working in Asian or Middle Eastern countries.
And it's the Brits working abroad that absolutely push for that term over all else. I've been in those communities and I can tell you it's not just Asia (by which most people generally mean South East Asia) and the Middle East, but East and West Africa and even mainland Europe are not exempt. (The retirement communities in southern Spain and the working communities in EU Brussels are particularly prevalent.)
Not just the Brits, of course, it's a general white economic migrant thing but that double standard (for me, as a Brit as well) is particularly telling.