I might be wrong but I think Carson's basic comment is that he wouldn't support, and he doesn't believe we should elect, a Muslim to serve as President. I'm not sure he ever said Muslims should be barred from running for that office on the basis of their religion, something which clearly seems unconstitutional.
What Carson said is literally no different than what was being said by many powerful people in this country
55 years ago. People have a right to question whether the President's strings are being held by the pope, a greedy catholic businessman from Hyannis Port, a mullah from the middle east, or just the wall street guys we already know about, and then vote accordingly.
Isn't that really our prerogative, as voters?