Retractile testicles is relatively common in men, and generally not an issue. Spanky is correct that there is a greater chance of torsion, so you should be very aware of any pain in either testicle, and seek immediate attention. (torsion can lead to loss of the testicle). There does not seem to have a noticeable effect on sperm efficacy either.
Normally this condition does not just suddenly appear, nor is caused by "pulling and stretching" of your penis. (BTW, if you were jelqing
, you should not do that, as it can create serious problems).
If the situation gets worse and causes symptoms, a urologist can do a minor procedure to tack the testicle to the scrotal wall to prevent this from happening (orchidopexy). However, this is rarely needed with a retractile testicle.