Maybe they are not happy with your work and they are using the “burden” as an excuse to fire you. Maybe the new gardener has a relationship with someone in the building who has persuaded the others that the new person is going to be better than you. That would be a conflict of interest.
If it was me and I really wanted the job, I’d be documenting what the new gardener is doing wrong then presenting that information along with a comparison between the new gardener’s fees and yours and suggesting to the Committee that they reinstate the previous relationship. Then make sure you attend the meeting so you can answer any questions and anyone who has a conflict of interest can’t bamboozle the Committee with incomplete or incorrect information.