Mervyn King said it was a 'delusion' that tightening regulation could stop banks' most risky activities from failing and leading to huge losses.
In his starkest warning to date, the Governor said banks may have to separate their day-to-day business from more speculative practices if they are to get state aid.
Mr. King also predicted that the United Kingdom's economy would grow in the remaining two quarters of the year.
He did not commit himself to saying the economy had come out of recession between July and September. Initial data on the economy's performance during that period is expected on Friday.
The Governor added that while it should become easier for households and businesses to borrow money, everone should be under no illusion that the path to sustained recovery will be smooth an painless, citing high unemployment and lower industrial output.
"We shall all be paying for the impact of this crisis on the public finances for a generation."In his speech to Scottish business organisations in Edinburgh, Mr. King hinted that G20 plans for tighter regulations may not be sufficient, because if banks knew they would be bailed out if they hit difficulties, they would continue to take risks.
"The belief that appropriate regulation can ensure that speculative activities do not result in failures is a delusion,"Banks have been criticised for making risky investments - which in previous years had brought hefty profits and large bonuses for their staff. However, when these investments went wrong, it contributed to the global economic crisis which saw several banks, including Royal Bank of Scotland and Lloyds, being part-nationalised.
Mr King suggested it was hard to see why taxpayer support could not be limited to retail banking.
"Anyone who proposed giving government guarantees to retail depositors and other creditors, and then suggested that such funding could be used to finance highly risky and speculative activities, would be thought rather unworldly. But that is where we now are."Direct or guaranteed investment in banks from the government was close to one trillion pounds, Mr King said.
"The sheer scale of support to the banking sector is breathtaking.
"Never has so much money been owed by so few to so many. And, one might add, so far with little real reform."
"It is hard to see how the existence of institutions that are 'too important to fail' is consistent with their being in the private sector.
"Encouraging banks to take risks that result in large dividend and remuneration payouts when things go well, and losses for taxpayers when they don't, distorts the allocation of resources and management of risk."
Mr. King said that those banks which continued to get public money to prop them up and aid their recovery should not be encouraged to try and earn profits to get out of government support by resuming the very activities that got them into trouble in the first place.