Dr. Mindy DeSeta states, 'Penis size is often treated like a shortcut - the bigger the penis, the more pleasure someone can supposedly give.' This cultural belief can significantly impact men's self-esteem and sexual confidence.
Altering your appearance (and perhaps life) because of a bad experience is giving that size queen way too much power. The first thing I want you to consider instead is therapy. Five inches is typically quoted as average in studies, though a recent meta-study of many studies of penis length found that the average has bumped up to six inches. That means even if you aren't quite average, you're within spitting distance of it.
Many studies show that most women don't care much about the length or girth of men's penises, either flaccid or erect. But men don't believe it. Sex educators talk themselves hoarse about this, but many men still crave huge ones, and many opt for enlargement pills (all frauds) or for costly surgery that might leave them deformed. A recent study explored women's preferences for dildos, penis-mimicking sex toys.
Your words are not going to make this OK. He needs to find some kind of solace within. It can be difficult to resist absorbing the preoccupations of a partner, but I think it's reasonable to distance yourself here. Chalk this up to a quirk or undesirable habit or hobby even and ask that he do his incessant measuring in private. This is his journey and you shouldn't get in his way.