Full credit goes to the original author, linked below. All blog posts were reposted either with permission of the author, or by anonymous submission by SAIRC members like yourself.
The Universal Approximation Theorem says a suitable neural network exists for any function — but not how to find it. This post explains training as exactly that search: starting from random parameters that produce garbage predictions, and letting an algorithm learn the solution from a dataset of inputs and labeled outputs.