Perhaps the Mikado had it right all along. Trouble is we're bound too much by ancient (some might say primitive) concepts, such as the biblical middah ke-neged middah (an eye for an eye, figuratively) but I suspect the real problem lies not with serious crimes or offenders but with criminality deemed 'trivial' by the Justice system but which the ordinary person finds not simply repugnant but utterly anti-social.
There's long been a theory of justice which extols the virtues of meeting minor criminality head on through retributive means as opposed to simply taking revenge (one extreme) or meting out minor punishment (the other). Thus the couple involved in verbally abusing a bus official might be deemed to have inflicted specifically mental and emotional damage, so would be treated in a similar way. The problems arise when trying to construct the system, which would have to be different for each criminal.
On the question of 'who pays' the field becomes much more confusing. In effect, we all pay for everyone who is hauled before the Beak because we all pay taxes. And let's not forget that's only the tip of the iceberg: we also all pay for the Tax collector, the Police, the education of their children (particularly galling if you don't have kids yourself, I imagine), the roads, the sand in the North shore (?), the swan droppings in the West, the rich who cleverly salt their hard-earned in off-shore trusts to deny the exchequer its dues, the people counting the goats, the bombs dropped on Afghanistan and so on. In fact, all in all the huge amount we pay towards our society dwarfs any tiny fine handed down to low level offenders.
As always, no easy answers.