I wasn’t being frivolous, I was deadly serious. I have never frequented a brothel but those hours seem very odd to me, why would you think it is a brothel? The noise could be from anything. You could send in a swat squad and find it is a little old lady who likes banging on pipes, it happened to me a number of years ago, when a little old lady did exactly that at 6am every day, until I asked her to stop.
More generally, as someone who has been employed to solve problems of various kinds for a long time, it has been my experience that people don’t try the most obvious and simplest solution first. When we have problems we are usually part of the solution, in that we need to take some sort of action to try and resolve the problem. The person next door is likely not aware that they are disturbing their neighbour – knock on the door and tell them that you are being disturbed by strange noises, can they assist? It is only if or when they refuse to assist that you then consider other options, although calling the police or Council re illegal brothels is the last thing I would do, in the absence of any clear evidence. They have better things to do.