This is just an opinion, I’m not a lawyer or expert.
I agree with the committee on this one. I cannot see that the owner who is living elsewhere and has rented out their lot has any rights to use the pool or any other part of the common property. As part of the lease agreement, surely the owner has conferred exclusive access and quiet enjoyment of the property to the tenant. This would include the apartment, the car parks (if any), and any/all common property which the occupier of the lot would normally have access to.