Well said re: gradient optimization vs. "getting slapped". However, note that since NN optimization is almost always nonconvex, we are NOT guaranteed to arrive an a optimal (or even close-enough) solution. A major limitation of gradient based optimization on nonconvex problems is that they are very susceptible to getting trapped in local minima.

But, for now it's the best tool we have, so we just have to hope that we get close enough, or just empirically run lots of times to find the best local minimum we can. Incidentally, this actually is more like a brute-force approach, but at the ensemble level, which is quite different than the article means it.