I don’t remember ever hearing “Brit” or “Brits” being used for immigrants in Britain and definitelly heard it used for when Britons are living abroad as immigrants (in fact when living in The Netherlands I had some colleagues who were in their own words “Brits”)…

Always thought it was just another way of saying British and not going into the specifics of which nation in Britain did people come from (the whole “nations” thing for people abroad is generally irrelevant or even misunderstood).