When I was 20 and fit as hell my dick seemed like it got bigger and harder. Over the years, I still get rock hard, but the max size is a bit less and a little bit more fat on my body probably makes it look smaller than back then as well.
I don't think it shrank, just not as ridiculously amped up like I used to be.