Have a look at this story about a Victorian couple who managed to block mass inspections of their home.
Then start taking action through Consumer Advice Victoria for a rent reduction backdated to when you first raised the issue.
And tell the landlord that you intend to stick to the letter of the law on how often they can show the property. Quite clearly they are asking too much for the unit and you are the one paying the price.
And finally warn the real estate agent that if they continue to bully you with threats of a “bad reference” you will report them to CAV and the Real Estate Institute of Victoria.
And have a look at this factsheet (it’s the online version of the one recommended by DaveB).