Full credit goes to the original author, linked below. All blog posts were reposted either with permission of the author, or by anonymous submission by SAIRC members like yourself.
The most mathematical post of the series, examining the Universal Approximation Theorem and why non-linear activation functions are essential to it. A glimpse into what computation inside a neural network actually looks like — rendering it, in the author's words, more prosaic and less magical.