Many of the comments seem to interpret this to mean "don't try to craft insights into your models. instead use dumb generic models and a lot of data." That's not how I read the article.

Not all convolutional neural nets do a good job at image classification, regardless of how much data you feed them. Dense neural nets are plain just plain bad at it. Not all brute force searches over chess plays win. Not all recurrent models translate text well.

The art of incorporating insights into model is still alive and well. Without introducing "structural biases" into models, none of our current methods would work. The problem is that we've managed to introduce a lot of structural biases without understanding them well. And we should do a better job of understanding these bits of side knowledge we introduce into models.

I also agree with the main point of the article that if you're at a crossroads between introducing a very blunt structural bias (like invariance under lighting for image classifiers), you're probably better off instead throwing compute at the problem (by, say, synthetically augmenting your dataset by 10x).