Penises don't actually shrink with age, at least not their hard size. What happens is that with age some men lose the ability to achieve a full erection, or at least an erection with the turgidity of what they used to achieve. It means its smaller when hard, but really only because they aren't hard.
The only things I am aware of that will technically shrink a penis are gaining weight and circulatory problems. If you've gained some fat that will accumulate around the base of your penis and make it appear shorter, estimates are 20-30 pounds will cover 1 inch of penis, though there is a lot of variation there. If you smoke or do anything else that negatively impacts circulation then you might have less blood flow. Guys who smoke typically have problems getting erections or getting as hard as they should, which makes it seem smaller.
Other than that I'm not aware of anything that would cause it to shrink. Your body will shrink as you get older, but that's due to losing bone density which obviously doesn't affect your penis. In fact it might make it look proportionally larger along with the effects of gravity.