I think you crossed the line when you went over and took a hold of his penis. It doesn't matter if you wanted to know what an uncircumcised penis felt like. How would you yourself honestly feel or react if your partner went and did something like this? If it would chew on your insides then you don't feel all that good about what happened with you. If you have no feeling then why would you be concerned about your partner having a reaction?
Whether or not you should tell him and the matter of guilt are entirely up to you, if this is a committed relationship as you state that it is then it should be able to weather any storm this could create. On the positive side you did not have further sexual contact with him so there was a level of restraint which might help you if your partner is extremely upset.
As for "should I just say "hey no problem, just watch yourself next time" "I think it's a little late for that or are you thinking that there will be a next time? I am sure the possibility of other men exposing themselves and even of men taking long leering looks at your penis are there, it's now up to you how you choose to conduct yourself. I have looked, and there are those who have looked at me. It does happen but the hands on contact is just not a good idea. (My view only, you do not have to agree with it).
Something else to keep in mind, there are men out there who will lure and bait with these tactics. To some degree even police officers use similar tactics (although I think they keep their trousers zipped up). The thing is there are men who get their jollies out of luring a gay man, getting him interested in the goods so to speak and then once the gay man unzips to join in, the first man (the instigator) will make a physical attack or have a friend or 2 waiting on the side who will make the attack. How would your partner have reacted if they had been forced to go to the morgue and identify your dead body? Maybe I'm being extreme but I'd rather have that than learn you were killed because of a foolish moment of sexual urge.
This is not to say I am a pure an innocent guy. I'm not. The fact is my partner Ray and I have a long time understanding of what we both want in life and in sex. I have had a few side adventures with other men, all of whom Ray knew about. We talked openly before hand on what was going to happen and in each case he knew the circumstances of the man I would be having an encounter with. There were times when during a talk it became clear he was not that comfortable with me being with a certain man and it was canceled on the spot. No questions asked on my part. I love Ray enough to respect his decisions and if he told me it was to stop permanently then I would go with that. He has allowed me to do somehting which very few men could be so open about. Nothing has been kept secret, a good relationship cannot survive on a stack of secrets. I could justify my escapades as being only oral sex and a little hands on action which is all they have been, there has been no anal sex in these matters and since it's this whole oral fixation of mine I could decide that it's okay to seek out a group of strangers and that that would be okay. The point is, my partner knows when I am doing anything with another man, I am not proud of what I have done and I am sure it has caused damage over time but Ray has always known (it's caused enough damage that I no longer even place the idea in front of him, the last few times were at his request and even that goes back some 2 years now). The saving graces are Ray's extremely large heart and patience.
Again the thoughts I have given are my view and reflect more on how I move through life.