Hi - I have been using Redmine for a couple+ years and I love it. I has all the same features of commercial, hosted services, but it's self hosted, you can easily customise it yourself by changing the code or thanks to the many plugins available, plus it has a really nice integration/view of the source code repositories (Git, SVN, etc).
On the public bug tracking, of course also with Redmine you can do this.
Another one that has not been mentioned yet but that I also like is ClockingIt (https://www.clockingit.com/).
It has some unique features and is available both as hosted service (amazingly, it's still free), and as downloadable app that you can install on your servers.
Before spending money with any of the commercial services, I'd recommend to give both these tools a try first, since both of them are very solid.