Neural networks stack affine transforms and nonlinear activations, then reduce loss by backpropagation and gradient-based optimization.