In my experience, three things related to changing science policy hit a nerve with researchers: funding cuts, restricted mobility of people, and general political instability. All of these aren't unique to the kind-of-hypothetical Brexit Effect; they're just brought to top of mind by it.
Funding cuts. It's no doubt that money and resources are the lifeblood of a research group - any group, for that matter - and whenever there's less money to go around it's an uncomfortable time, by definition.
There's a wonderful post at the London School of Economics' BrexitVote blog that goes into extensive detail about how the UK science funding may change. Specifically on how whatever gains won by leaving the EU are overshadowed by the general loss of economic activity if the UK were to go through with the plan:
Importantly, even the more optimistic assessments of the UK’s economic performance following a Brexit ,.. model an immediate loss in GDP for the transition years following a Brexit. The size of that loss is substantially larger than the current net contribution of the UK to the EU budget. ...Other parts of the post go on to explain how Switzerland manages to get along with science just fine without being an EU member.
Therefore the attempt to financially gain in the short term via a Brexit is akin to killing the goose that lays the golden egg. It is a sure-fire short term loss, wiping any free money for Research & Innovation investment until at least a decade down the line – according to the most optimistic scenarios. This strongly counters any claim that voting to leave the EU provides immediate funds for a shot in the arm of national science. The extra money simply will not be there for science as the UK economy is hit by huge transition costs.
Now you might point out that the above scenario explains what would happen to global research funding (similar to a US-style Sequestration effect), and ask what about cuts to specific programs or areas of research? This, sadly, is sometimes needed. Just as in any other business, as the needs of the 'market' move on, science needs to adapt to ask new questions, use new technologies, and re-train people.
This process of creative destruction has been accepted by mainstream business people and spawned so many different bestsellers (including the classic ones by Eric Topol and Rotman Professor Sarah Kaplan). It really should be one of those things that's accepted as a given by grant funded researchers, not fought against.
Next post, Mobility of People.