A campaign group says its members have had unannounced visits and emails saying they could be moved far from London.