Im not absolutely sure of the law in Victoria but based on my experience in NSW I provide the following comments.
It appears that you did not correctly call a meeting. Just sending an email in itself is not a meeting notice. Therefore what you have taken is a poll. Ask your strata manager to correctly call a meeting.
If a meeting is called correctly, then the defence of “I didn’t receive the notice” is invalid. Provided the correct form of delivery is used (post or email) and the correct days allowance for delivery has passed, then a notice of a meeting is deemed to have been sent and delivered.
Secondly the Owners Corporation (or whatever it is called in Vic) has an absolute obligation to maintain and repair its building. there is no debate in this matter from any owner
Thirdly, usually for a special levy it does not require a unanimous decision. usually its just a majority thats required but check in this case wit your strata manager.
The recalcitrant can protest all they like, but if the majority agree on a special levy then so be it.