Why is Colin Baker regarded as rubbish in his role as Doctor Who?

Micheal Grade called him "utterly unlikeable" and "god-awful". It might be worth noting that even though at the time, Colin Baker's stint was seen as one of the shortest as Dr. Who, but Colin Baker still outlasts Chris Ecclestone, and David Tennant for that matter.