Neural network basics
What a neural network is, how it learns by gradient descent, and how to train one properly in PyTorch. The unit ends with a first field surrogate, tested on designs it has never seen.
Start the unit From one neuron to a trained field model
- 4.1Neurons, layers and MLPsA neural network is a flexible regression model built from simple layers.
- 4.2Loss and gradient descentTraining is walking downhill on the loss, a small step at a time.
- 4.3Backpropagation and autogradThe chain rule gives the slope for every weight at once; PyTorch computes it for you.
- 4.4Normalising inputs and outputsPut every number on a similar scale before training, using statistics from the training set only.
- 4.5The training loop in PyTorchBatches in, forward, loss, backward, step; check the validation loss as you go.
- 4.6Overfitting and regularisationRead the learning curves, and stop the network memorising its training runs.
- 4.7A first field surrogateA network that takes a point and a design and returns the field value at that point.
- 4.8Testing on unseen designsCompare surrogate and solver on designs the network never saw, inside and outside the training range.
- 4.9Your first surrogate, end to endDownload real solver runs, split by design, normalise, train, test and report: the whole recipe in one script.
You can thenTrain and validate a small network end to end.
Read and build
ReadNeural networks3Blue1Brown · video lessons ↗Learn the basicsPyTorch tutorials ↗Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional DomainsTancik et al. · NeurIPS 2020 ↗
BuildTrain a neural field that maps a surface point and the angle of attack to pressure in PyTorch, and plot its learning curves.
Basics Core Advanced Frontier