I agree Hugo, HMRC are doing less and less investigation work these days. I think it is political as the Tories don't like their cronies to be investigated. This started with Thatcher who ran down what was the Inland Revenue as the working conditions became intolerable and very many people left. Even when investigations were the flavour of the month, the big boys were left untouched as they didn't have the time, or resources, to take them on. Just a minority of large cases were tackled but this was by specialist teams in regional offices. Of those, even the cases where there was massive fraud, just a tiny number were prosecuted.
These days there is less interest in tackling investigation work and the tax loss must be enormous. It is a different story with people who claim benefits fraudulently as they are prosecuted frequently. As you say, it is us and them.