Ten equations every AI student should know Image: Smithsonian Institution, public domain

Ten equations every AI student should know

A working researcher in this field carries perhaps a dozen equations in their head and derives the rest when needed. Here are ten of them, in the order the AI Masterclass meets them, each with what the symbols mean and where it is used. Three ideas account for nearly all of them: a weighted sum, a probability that is updated by evidence, and a step against a gradient. If you can read these ten, you can read most papers' methods sections.

The ingredients

1. The dot product

\[ \mathbf{w}\cdot\mathbf{x} = \sum_{j=1}^{d} w_j x_j \]

Multiply two lists of \(d\) numbers component by component and add. As a weighted sum of the inputs \(\mathbf{x}\) with weights \(\mathbf{w}\), it is what a neuron computes before its activation function. As a measure of how much two vectors point the same way, it is the similarity behind embeddings and the score behind attention. A matrix-vector product is many dot products at once, and a GPU is a machine for doing billions of them a second.

2. Bayes' rule

\[ P(A \mid B) = \frac{P(B \mid A)\, P(A)}{P(B)} \]

The probability of a hypothesis \(A\) given evidence \(B\): the prior \(P(A)\), multiplied by the likelihood of the evidence under the hypothesis, divided by the overall probability of the evidence. A disease with prevalence 1 per cent and a test that is right 95 per cent of the time give a posterior of 16 per cent after one positive result, as the article on the rare disease shows. Naive Bayes, Bayesian networks, and the probabilistic reading of every classifier's output come from this line.

Learning

3. The mean squared error

\[ L(\mathbf{w}, b) = \frac{1}{N}\sum_{i=1}^{N}\left(y_i - \mathbf{w}\cdot\mathbf{x}_i - b\right)^2 \]

The average squared gap between the \(N\) labels \(y_i\) and the predictions of a linear model. It is the first loss function a student meets, and the template for all of them: a single number that says how wrong the model is on the training data and that can be differentiated with respect to the parameters. For a straight line, setting the derivative to zero gives the least squares formula; for everything else, the next equation takes over.

4. The gradient descent update

\[ \boldsymbol{\theta} \leftarrow \boldsymbol{\theta} - \eta\, \nabla_{\boldsymbol{\theta}} L \]

Move every parameter \(\boldsymbol{\theta}\) a small step, of size set by the learning rate \(\eta\), against the gradient of the loss. This one line trains linear regression, logistic regression, a convolutional network, and a language model with hundreds of billions of parameters. In practice the gradient is estimated on a random batch, the learning rate follows a schedule, and Adam rescales each parameter's step, but the line does not change. The article on the gradient walks three steps of it.

5. The cross-entropy loss

\[ \ell = -\left[y \log \hat{y} + (1 - y)\log(1 - \hat{y})\right] \]

The loss for a classifier that outputs a probability \(\hat{y}\) that the label \(y\) is 1. It is the negative logarithm of the probability the model assigned to what actually happened, so minimizing it is maximum likelihood, and its gradient with respect to the weights of logistic regression is the simple \((\hat{y} - y)\,\mathbf{x}\). With a label among 50,000 tokens instead of 2, the same formula, \(-\log \hat{y}_{\text{correct}}\), is the pretraining loss of every language model.

6. The softmax

\[ \mathrm{softmax}(\mathbf{z})_k = \frac{e^{z_k}}{\sum_j e^{z_j}} \]

Turns any list of real numbers \(\mathbf{z}\) into probabilities: all positive, summing to one, with the largest input getting the largest share. It is the last layer of every classifier and the step inside attention that turns scores into weights. Divide \(\mathbf{z}\) by a temperature before applying it and the distribution sharpens (below 1) or flattens (above 1), which is the dial behind the creativity setting of a chatbot.

Acting and attending

7. The Bellman equation

\[ Q^*(s, a) = \mathbb{E}\left[r + \gamma \max_{a'} Q^*(s', a')\right] \]

The value of taking action \(a\) in state \(s\) and acting optimally afterwards equals the expected immediate reward \(r\) plus the discounted (by \(\gamma\)) value of the best action in the next state \(s'\). It is a consistency condition that the true values must satisfy, and turning it into an update rule gives value iteration and Q-learning. The deep Q-network that learned Atari games, described in the article on Atari, is a network trained to satisfy this equation.

8. Scaled dot-product attention

\[ \mathrm{Attention}(\mathbf{Q}, \mathbf{K}, \mathbf{V}) = \mathrm{softmax}\!\left(\frac{\mathbf{Q}\mathbf{K}^{\top}}{\sqrt{d_k}}\right)\mathbf{V} \]

Each query is compared with every key by a dot product, the scores are scaled and turned into weights by the softmax, and the output is the weighted sum of the values. Stack it eight times in parallel, add residual connections and a feed-forward layer, repeat the block, and you have the transformer of 2017, which is the architecture of every large language model. The article on attention works it on three tokens.

Scale and alignment

9. The Chinchilla rule

\[ C \approx 6\,N\,D, \qquad D \approx 20\,N \]

Training a model with \(N\) parameters on \(D\) tokens costs about \(6ND\) floating point operations, and for a fixed budget the loss is lowest when the tokens number about twenty times the parameters. The second part is an empirical fit from 2022 and holds within the range of models it was fitted on; the first part is arithmetic. Together they turn a compute budget into a model size and a dataset size, and they explain why models stopped growing in parameters for a while and grew in data instead.

10. The RLHF objective

\[ \max_{\boldsymbol{\theta}}\; \mathbb{E}\left[r(y)\right] - \beta\, \mathrm{KL}\!\left(\pi_{\boldsymbol{\theta}} \,\|\, \pi_{\mathrm{ref}}\right) \]

Tune the model \(\pi_{\boldsymbol{\theta}}\) so that its answers \(y\) score highly under a reward model \(r\) learned from human comparisons, while a penalty of strength \(\beta\) on the Kullback-Leibler divergence keeps it close to the reference model it started from. The penalty is the leash: without it the model finds answers that fool the reward model and stop being language. This objective, or its reward-free cousin direct preference optimization, is how a text predictor becomes an assistant.

What is missing

Backpropagation is missing because it is the chain rule applied to all of them rather than an equation of its own. The convolution, the LSTM gates, the policy gradient, and the sparse autoencoder loss each deserve a place and did not fit. And every equation above is a simplification of the practice: gradient descent does not find the minimum, the 20 tokens per parameter rule has conditions, and attention is not, despite the title of the paper, all you need. The masterclass's key equations page collects the full list from all sixteen units with those conditions attached, and the curriculum says which unit derives each one.

Learn how machines learn, properly

The AI Masterclass: weekly units, quizzes, a final exam, and a certificate.
About the program