Phone Hacking - How do they know?

How does one know one's phone has been hacked?
Surely a reasonable question to ask, and one for which I have so far failed to find an answer.
In the recent cases, it was not so much the phone that was hacked, but the voice mail which are recordings held by your phone provider containing messages from other people who were unable to speak to you directly. I don't own a mobile phone, but my daughters both tell me that they listen to these messages using their mobile phone following which they erase them.
It is a simple procedure, and you need a password in order to hear the messages.

But my question still remains, how do you know someone has been listening to your messages? Surely unless the person leaving the message can confirm that they have told no-one else the contents of the message, and you don't subsequently tell anyone of the content, you only know it has been hacked if some third party also has the information. Research amongst family members failed to reveal any single item of information which they could honestly claim was only known by them and one other person.

Now, I'm not disputing the fact that phones have been hacked, but in view of the ever growing list of failed politicians and "celebrities" who are making the claims, I am becoming more sceptical about the whole affair. Of course, politicians, as has been shown, will do anything for money, whilst these so-called "celebrities" (many of whom I have never heard of) will say anything for publicity (and were probably stupid enough to leave the default password on their phones).

But in spite of all the column inches and extended news bulletins, my basic question "How do they know" remains unanswered.