Here is the way that i look at it.
Refs MUST have unconcious bias, simply due to numbers and their own wellbeing
Imagine making a bad call against Man United - billions of worldwide "followers" and possibly millions of fans in the UK, the social media threats, abuse, then in the street etc.
Compare that to making a mistake against Brentford for example - a million followers at most across the World and 50,000 fans maximum across the UK, little chance of being spotted in the street etc.
Refs are normal people like you and I, but in a profession where they can be subjected to unwarranted and unneeded harrassment and hounding over a mistake.
You can see you they are a lot more hesitant to give a major decision against the big teams than they are against smaller ones.