When I was in Australia, one of my co-workers asked me what it was like to live in Boston. He commented on the fact that Boston was famous for cheese. I cocked my head, and thought...is he confusing Philadelphia Cream Cheese with Boston Cream Pie? I was bit lost, and he grew more frustrated with me. "Cheese! You know...cheese!" I just didn't get it. Finally he said "The TV show! with Sam and Norm, the bar! Cheese!" I felt so stupid.
That's what usually gets me...context. If I understand the basic flow of the discussion, I can follow an accent. If the subject is unknown to me, or I don't make the connection...such as Boston cheese, I'm pretty dense with accents.
I might be clueless with them...but I really enjoy accents and 90% of the time find them fascinating and enjoyable. When staying in Australia...I began to think to myself with an accent. Theirs, to me, is a very lazy accent that indulges the tongue and I find it quite familiar being from the American South (which I also think of as a lazy mouth accent). Here in the South, accents tend to tie closely with socio-economic groups. Those with thicker accents are generally more vernacular and less mobile. There are instructional classes here to teach people to temper their accents in order to become more attractive to potential employers who might have lower expectations for those with heavy accents. In the context of my region I have no accent at all...and people often assume a host of things about me because of that.
The fascinating world of linguistics!