Simple because it's the only county where the name of the city and the county are the same (that I can immediately think of), to differentiate between the two.
For example, most other counties have the county town name suffixed by -shire (Yorkshire, Cambridgeshire, Hertfordshire, Leicestershire, Oxfordshire etc.) or an abbreviation of it suffixed by -shire (Lancashire, Hampshire). Some other counties have no relationship between the county town and the county name or are based on archaic counties (Cornwall, Kent, Cumberland/Cumbria, Northumberland/Northumbria, Norfolk, Suffolk) or have lost the -shire in common usage (Devon/Devonshire).
There have long been arguments that signs such as 'County of Cambridgeshire' are incorrect, as shire means 'county of' - that signs should say either 'Cambridgeshire' or County of Cambridge. Durham is the only county that actually gets it right.