Just because something has been put in a skip/dumpster/bin, it's not 'abandoned' by the original owner, therefore it's not open to anyone else to come and take it.
Tax-avoidance is arranging one's affairs within the scope of the tax-code to minimise one's liability to pay tax. It's perfectly legal and - unless you in some warped way believe that the State knows better how to spend your money than you do - perfectly moral.
It's tax-evasion which is the illegal thing. To quote: