If your neighbour has a fixed term residential tenancy the agent has to give 7 – 14 days written notice of an inspection. That agreement doesn’t mention whether he has to be at home, but a time span of 2 hours has to be nominated (unless in remote area). As I see it, there’s nothing to stop the agent or anyone for that matter looking at the front, but if they go around the back they’re most likely on the lot and could be considered trespassing.
The agent seems somewhat naive in admitting to a surprise inspection and discussing the state of the premises with you.
I’d be inclined to just let your neighbour know what you observed and let him take it up with the real estate agency and/or Tenants Union if he is unsatisfied with their response.