I think this just comes down to human nature. Give people (engineers, execs) a metric that looks like a good proxy for performance and they will overcommit on that metric as it is a safer bet than thinking outside the box. I think the incremental improvements in deep learning with all those benchmarks are a similar situation.

That’s why we need rules and regulation around gatekeeper platforms like iOS and android. If both demand 30% there is no choice not to pay in certain circumstances.