Fiscal conservatives aren't wackjobs, they're just laboring under the misconception that because a car is designed to move forward it shouldn't also have brakes.
I actually think there's a bit of truth to the joke. To me, libertarianism is what comes of reading only one book (taking for granted that most libertarians have read many books, but all those books are the same book). Libertarians read Hazlitt or Rothbard or, if they're really smart, von Mises, and they notice the huge gap of thought between those authors and naive fiscal liberalism, and so they think they've ascended the mountain and are looking down on the liberals below mired in ignorance.
What they don't realize is behind them is another even steeper mountain, (that second road begins, delightfully ironically, with Hayek, who is utterly misunderstood by libertarians, and goes right through Adam Smith) and at the top of that mountain of knowledge one is back at liberalism, sophisticated this time. Conservatives continually assign the motives and thinking of naive liberals to sophisticated liberals, and thus never speak to the real issues.
But the smart ones get there eventually, this recovered fiscal conservative can promise you.