Credit cards make money on each transaction. If it's not the interest they want, it's the transactional VISA fees on $2,400 that will make them a few bucks even if the balance is paid immediately - which of course is to your benefit.
That said, I would request they waive the fee or I would close it. I agree with the poster who mentioned Credit Unions. 2.9% and no fees on my CU VISA. Other VISAs and MCs that have tried to nail me with inactivity fees either waived them or I closed the account. I try to charge something small on the cars at least every other month (tank of gas or grocery shopping) and pay it off immediately.
One note of caution... closing a credit card can HURT your credit score. One aspect that weighs in on your CS is credit utilization rate. The more of your available credit that is used, the lower your credit score is. If you close this card and you have outstanding balances on other cards, you're suddenly that much closer to your total credit "ceiling". However, if you're not going to be in the market soon for a home, car or other big credit purchase, close the account and work on paying off your other balances. Your CS will recover.