Is it possible to change the shape of the ad? If you combine the odd shape of the ad (odd shape because of the placement) and add that with the occasional announcement that appears at the top of the page, it turns the site into a scroll fest.
I'm not really sure how much engagement the ads get, but if it's low, then I don't think there is anything adverse in changing the ads to a rectangular banner ad, if there is ability too.