It hasn't been "this way forever." Until 1975 commissions were fixed by the NYSE. Violation of the "kosher eighth" was punishable by expulsion. Hence the proliferation in the 70's of block houses like Jefferies. Institutions basically told the NYSE, take those commissions and shove 'em. Ever wonder why trading went from 2 million shares a day to 200m a day in several years? Secondly, if you service a large institutional account the proceeds are TREMENDOUS. In 1992 I was at a low ebb trading at the CBOT. By a stroke of luck I hooked up with a portfolio manager at a well known private bank. I brought a portion of his business to a Primary Dealer (Treasury) and moved to NYC. My payout was 40% of gross. I had a $30,000 day from this single account. Granted, being a bond broker is better business than equities. However I didn't get the impression in Wall Street that Bud Fox was a millionaire. If you remember he needed to puke his 950k condo in the midst of his blow-up with Gekko.
True when they deregulated commissions the entire business changed. I don't however remember anyone charging anything near 5%. Pabst even guys who cover State Farm, Nationwide, and All State on the fixed side don't live like Bud Foxx.
LOL, funny you mention those insurers. I covered a trader at AllState for a while. A total bust. I printed the guy ONCE!