Soldiers and militia is a different matter. Those we're the civilian collaborators.
The main reason General Montcalm or whatever wanted to go there was to pave for a greater invasion of Canada. His death, and the mass route of his force ended it.
"The city's garrison, a motley assortment of regular troops and militia led by Quebec's provincial governor, General Guy Carleton," It wasn't just British Regulars.
Better the devil you know. And please, the Americans continued the trend of British exploitation on there own very, very well. Manifest destiny ring a bell?