No, I don't have any suggestion here.
The issue is that there is a closed loop control system at work, i.e. a form of negative feedback. Your brain has an idea of how much testosterone there should be in your blood (a set point) and measures the actual level to compare and uses that to work out how much of the hormone GnRH to make. That in turn works on the pituitary gland which in turn makes the hormones LH and FSH. Of those LH works on the Leydig cells of the testicles to cause them to produce testosterone.
When there is no external source of testosterone, or anything else that is similar enough to be mistaken for it, this means if your testosterone level is below the set point the brain makes more GnRH, the pituitary makes more FSH and LH so the testicles make more testosterone and your testosterone level rises. If the level is too high, the brain makes less GnRH, the pituitary makes less FSH and LH, the testicles make less testosterone and your testosterone levels falls until it is back at the set point.
This just like a more complicated version of how thermostatic heating works.
Now, if you testosterone level is low (compared to what doctors think it should be), there can be two reasons for that:
1. The set point in your brain is low, i.e. your brain is quite satisfied with a low level.
2. The set point is normal but your testicles (or the pituitary) are not so efficient.
For no. 2, testosterone supplementation would not have much effect on your testicles because you would not have raised your blood testosterone to above your brain's set point.
For no. 1, though, supplementation will bring your blood testosterone level up above the set point in your brain so you brain will make less GnRH to try to get it down again. That means less FSH and LH and less FSH means less stimulation to the seminiferous tubules that make up most of the volume of your testicles. Like most parts of the body the testicles adapt to the demands placed on them so with little hormonal stimulation they reduce in size.