Like all of our physical features, it is and it isn't herditary. To some degree, all of our physical features will bear a resemblance to our parents' features, and the odds are that we'll be simliar to them in certain ways. However, saying that we're likely to resemble our parents doesn't mean we actually will resemble them.
Early in the study of the field of statistics, an English mathematician measured the heights of a number of fathers and sons. He found that, on average, the sons of men who were taller (or shorter) than average also tended to be taller (or shorter) than average - but the sons tended to be not as tall (or as short) as their fathers, and in some cases the sons actually were shorter (or taller) than average. That led to the formulation of the statistical principle called regression to the mean, which states that if you measure a quantity in a group of some sort, as you make more measurements over time the average value of that quantity will tend to approach the average value for everyone as a whole. If we think about this in terms of penis size, a man with a large penis is more likely (but not guaranteed) to have a son with a large penis, but if we extend this chain through future generations, the odds are likely that his descendents will have smaller penes (and conversely, as one goes back to the man's ancestors, they likely had smaller penes, too).
Let's also remember that only half of our genes come from our fathers. To use the distaff side of this example, my wife's father was 6' 2" tall and her mother is 5' 9" tall, both of which are well above average height. It might not surprise you to know that my wife is 5' 11", also well above average. However, she shares almost none of her physical features with her mother; for example, my wife's breasts are huge while her mother is as flat as a board. Among her ancestors, it appears that the women with bigger breasts tend to be on her father's side of the family, so it looks like she got her big-boob gene from her father rather than her mother. Similiarly, the son of a man with a big penis might still have a average or a small one if the son got his penis-size gene from his mother's side of the family.
The short answer, I guess, is that while it's more likely that the ancestors or descendents of a man with a big penis will also have a big penis, there's no guarantee that it will be as big or even big at all.
