You can do some investigating on your own, wether with the intent to catch him in the act, or to alleviate your own irrational fears that he is cheating. This is what you need to do. First, check your phone records and the records of his cell phone calls. This can be done easily online or using the Caller ID option on your home phone and the Received Calls on his cell phone. You can look at it while he's in the shower, write down the numbers, and he won't ever know.
Just doing this little bit of research may be enough to put your mind at ease. The best scenario is that you will find nothing to implicate your husband in any affair. It was all in your head, you can feel better now. What happens though if you see a frequently called number that you don't recognize? What do you do then?
You then will take that number to your computer and do a free reverse phone number lookup on it. Some basic information about that number is provided with a free service, for instance the city and state that the number originated from - if you want more information, you need to pay (just once) for the detailed information.
The full report will give you information like a name and address that the number belongs to, including a satellite image and possibly even names of family members and neighbors. Does this seem like you're digging too deep? What if her phone is in her husband's name and you don't recognize it? It would be nice to know the women's names who might use the number, too.
Once you have the information you needed from performing a reverse phone number lookup, it is up to you to decide what to do with that information. You should surely keep it legal - but you don't have to keep it nice.