Matt – if you’re in NSW, a strict interpretation of Sect 40A of the Strata Schemes Management Act would suggest that your “caretaker” is actually someone who’s been engaged by your Owners Corporation (O/C) to perform cleaning and maintenance duties; somewhat akin the Janitor to you glibly referred to in your post.
Whatever the nature of the person’s appointment, it should have, at least initially, been formalised in some way such as via a a scope-of-work or even a Caretaker Agreement under Sect 40A, and your O/C or Strata Manager should only be making agreed payments in accordance with that and/or on the basis of submitted invoices.
Whilst your O/C could terminate its “caretaker” by a simple majority resolution taken at a General Meeting, I’d suggest that in the first instance you ascertain the terms of their appointment, any conditions attached to that such as them holding an ABN, liability insurance, and properly invoicing for services completed, and critically if there are any formal termination provisions in place.
That information should indicate how you and like-minded Owners should best proceed, but feel free to come back if you then need any further information or advice.