In Bayesian deep learning, we infer a posterior distribution over the weights of the network. This provides a variety of improvements over typical neural networks, whose outputs come from maximum likelihood (point) estimates of the parameters. One of the first things I wondered when reading the seminal paper was "why a Gaussian?". Yeah, yeah, most things are distributed...