My experiences, for whatever they're worth:
Viagra works harder, so to speak, and for about 12 hours. Side effects are more pronounced (flushed face, nasal congestion) and it is very sensitive to food intake prior to dosing...any fat in a meal reduces effectiveness substantially. The day after, I don't feel any after effects at all.
Cialis doesn't get me rock hard, but lasts the full day-and-a-half, and does seem to sort of linger as you've described with Viagra. Food intake has no effect whatsoever.
My guess is that both may not fully metabolize for several days, so some is still circulating in the bloodstream. Whether it's enough to cause some effect depends on individual responses.