Given his popularity, he kind of functions as a legitimate critic at this point. Other critics don’t really have more or less credence than he does, the only difference is that they write their opinions for a magazine while he writes his for a several-million subscriber YouTube channel.
Plus, I don’t really feel like saying someone is “contrary to what other critics say” is a fair point… critics give wildly different reviews to the same album all the time, and nobody calls for their removal from the site solely based on that.
Saying that he “influences ratings of actually good music in a bad way” suggests that there is objectively good music and thus objectively bad ratings, which kind of goes against the whole point of AOTY, doesn’t it?