Powers and logs, functions that chain, sums that grow, and the row-times-column move at the heart of every neural network — with a number line to check each step yourself.
The same three numbers can make two different answers; brackets decide which. Computers follow the contract exactly, so the order is a rule to write down, not a preference to argue about. And a minus sign in front of a power applies after it: −3² = −9.
2 + 3 × 4 = 14 · (2 + 3) × 4 = 20f(x) = x² turns 3 into 9 and never changes its mind. Chaining machines is composition — run g first, then feed its output to f — and swapping the order changes the answer. A neural network is just this, chained very deep.
f(g(2)) = 49 ≠ g(f(2)) = 13Lay one row of the left matrix on one column of the right: multiply the pairs, add them up. The inner dimensions must agree — (m × n) @ (n × p) = (m × p) — and the outer two survive. Nothing else in matrix multiplication is happening.
(2 × 3) @ (3 × 2) = (2 × 2)Every lesson ahead is built from four small moves: apply a rule, raise to a power, add things up, and lay a row on a column — nothing else is allowed to stay mysterious.
By the end you will read every symbol in this notebook aloud, move between powers and logs, expand a Σ into its terms, and compute a 2 × 2 matrix product by hand — the arithmetic every later lesson assumes.
Each symbol replaces a sentence. Once you can say them aloud, formulas stop looking like a wall — and underneath every one of them is arithmetic you already know.
Start with the rule that trips up more people than anything else: operations have an order. Multiplication and division run before addition and subtraction, and brackets run before everything. 2 + 3 × 4 is 2 + 12 = 14, not (2 + 3) × 4 = 20. If you want 20, say so with brackets.
The other small trap is the minus sign. −3² means −(3²) = −9: the power happens first and the sign applies after. To square a negative, wrap it: (−3)² = 9. It is worth knowing this once, because a missing bracket in a loss function produces a wrong number, not an error message.
Compute the expression the maths way and the strict left-to-right way. Watch the two markers land in different places.
× and ÷ bind tighter than + and −, so they run first. Brackets run before everything. The slider lets you walk one operation at a time.
| Symbol | Read it aloud as | Example |
|---|---|---|
Σ (sigma) | “add up all of these” | Σᵢ₌₁³ aᵢ = a₁ + a₂ + a₃ |
Π (capital pi) | “multiply all of these” | Πᵢ pᵢ = p₁ · p₂ · p₃ |
aᵢ (subscript) | “the i-th entry of a” | a = [4, 7, 9] gives a₂ = 7 |
x², xⁿ | “x squared”, “x to the n” | 2³ = 2·2·2 = 8 |
|a| or ‖a‖ | “the length of a” | |[3, 4]| = 5 |
a · b | “a dot b” | a dot product (Lesson 1) |
A @ B | “A matmul B” | matrix multiply (chapter 05) |
Aᵀ | “A transpose” | rows and columns swapped |
A⁻¹ | “A inverse” | the matrix that undoes A |
≈ | “approximately equals” | π ≈ 3.14 |
→ | “maps to” or “approaches” | f: ℝⁿ → ℝᵐ takes n in, gives m out |
argmax | “the input that gives the biggest output” | argmax of [0.1, 0.7, 0.2] is index 1 |
P(A|B) | “probability of A given B” | the bar means “given that B happened” |
E[X] | “expected value of X” | the long-run average (Lesson 6) |
e | “Euler's number”, ≈ 2.71828 | exp(x) means eˣ |
ln x, log x | “natural log of x” | the power you raise e to, in order to get x |
The symbol says: let i walk from 1 to 3, and for each i multiply aᵢ by bᵢ, then add the results. With a = [4, 7, 9] and b = [2, 1, 3]:
Notice there is no cleverness here: expand the symbol into its terms, compute each little product, add them up. That move — multiply pairs, then sum — is the dot product of Lesson 1 and the engine inside matrix multiplication in chapter 05.
What does 2 + 3 × 4 equal?
Number in, number out, the same output every time for the same input. That is the whole definition — everything else in this notebook is machines feeding machines.
f(x) = x² turns 3 into 9. The letter f is the machine’s name, x is the slot you feed, and x² is the rule the machine applies. A machine can take several inputs at once: f(x, y) = x² + 3xy + y² turns the pair (1, 2) into 1 + 6 + 4 = 11.
Composition chains two machines: f(g(x)) means “run g first, then feed whatever comes out into f.” If g(x) = 3x + 1 and f(u) = u², then g(2) = 7 and f(7) = 49, so f(g(2)) = 49. Swap the order and you get a different machine: g(f(2)) = g(4) = 13. Order is part of the meaning, exactly as it will be for matrix multiplication in Lesson 3.
Pick a rule, drag the input, and watch the output move along the curve. Tick “chain” to feed the output back in as the next input.
A function is a machine: one input, one rule, one output, the same output every time. Chaining two machines is composition — and the order you chain them in changes the result.
A second example, small enough to do entirely in your head. Take g(x) = 2x + 3 and f(u) = u − 1, and feed in 4:
The inner machine always runs first. When in doubt, evaluate the brackets from the inside out, one number at a time, and write each intermediate down.
With g(x) = 3x + 1 and f(u) = u², what is f(g(2))?
An exponent counts repeated multiplication; a logarithm asks the reverse question. These two appear in every lesson from 4 onward, so they are worth getting right once.
2³ means 2·2·2 = 8: the little number counts how many copies of the base are multiplied together. A logarithm runs the question backwards: “2 to what power gives 8?” The answer is 3, written log₂ 8 = 3.
In machine learning the base is almost always e ≈ 2.71828 — Euler’s number — and that logarithm is written ln or simply log. So when a paper says “log probability”, read “natural log”. The reason for e is calculus: eˣ is the one function whose slope equals its own value at every point (Lesson 4), which keeps derivatives clean.
Move the exponent and watch the same pair of numbers appear on both sides: 2ˣ builds the value, log₂ takes it apart again.
A logarithm asks the reverse question: “2 to what power gives this number?” In machine learning the base is usually e ≈ 2.718, but the idea is exactly this one.
Second worked example: a three-word sentence. A model assigns probabilities 0.9, 0.8 and 0.7 to three words in a row. The probability of the whole sequence is the product 0.9 · 0.8 · 0.7 = 0.504. In log space: ln 0.9 ≈ −0.105, ln 0.8 ≈ −0.223, ln 0.7 ≈ −0.357, and the sum is −0.105 − 0.223 − 0.357 = −0.685. Check: e^(−0.685) ≈ 0.504 ✓. Same number, but a long sentence adds up instead of multiplying down towards zero — the sum never underflows.
Given ln 2 ≈ 0.693, what is ln 8?
A list of numbers is the most common object in all of machine learning. Two moves cover almost everything: operate entry by entry, and add up a sequence of entries.
An array (or list, or vector) is an ordered stack of numbers: a = [4, 7, 9]. The little letter below a symbol selects one entry — aᵢ is “the i-th entry of a”, so a₂ = 7. One warning worth internalising now: maths usually counts from 1, code always counts from 0. In the same list, maths calls the last entry a₃ = 9 while Python calls it a[2] = 9. When a sum runs off the end of a list in code, this is almost always why.
Element-wise operations move entry by entry: add matching positions, or stretch every entry by the same number. The length of a vector is Pythagoras run across its entries: |[3, 4]| = √(9 + 16) = 5. Dividing a vector by its length leaves a unit vector pointing the same way: [3, 4] / 5 = [0.6, 0.8]. Lesson 1 uses that constantly.
The index i walks 1, 2, 3 … and the running total grows by one bar per step. Press play or drag the stop-at slider.
The index is the counter inside the Σ. The note under it says where the counter starts; the letter on top says where it stops. In machine-learning code, sums like this run over examples.
The dot product is the most-used sum in the notebook. It multiplies matching entries and adds the results. Read it twice with real numbers so the symbol has no mystery left:
The code version of the same sum is one line: sum(ai*bi for ai, bi in zip(a, b)). The Σ is the same instruction; the loop is just written down in symbols.
| Python you will read | Plain English | Example |
|---|---|---|
x = [4, 7, 9] | a list, in order. Python counts positions from 0: x[0] = 4, x[2] = 9 | len(x) = 3 |
[f(x) for x in xs] | a new list made by applying f to each item | [x*2 for x in [1,2,3]] → [2, 4, 6] |
zip(a, b) | walk both lists side by side, in pairs | list(zip([1,2],[3,4])) → [(1,3), (2,4)] |
sum(a*b for a,b in zip(x,y)) | multiply the pairs, then add: a dot product | x=[1,2], y=[3,4] → 1·3 + 2·4 = 11 |
A @ B vs A * B | matrix multiply vs element-wise multiply in NumPy | the silent bug of Lesson 2 |
Π is the multiplication sibling of Σ. It shows up when events do not influence each other: the probability of two heads in a row is ½ · ½ = ¼, a Π over two coin flips. Lesson 6 builds everything from that one fact.
What is Σᵢ₌₁³ i²?
A matrix is a grid of numbers. Multiplying two of them sounds intimidating and is not: every entry of the answer is one row laid on one column, exactly the multiply-and-add you already do.
A matrix is a rectangular grid of numbers. Its shape is written rows × columns, always rows first: a 2 × 3 matrix has 2 rows and 3 columns. The entry in row i, column j is written A[i][j] or aᵢⱼ. A vector with n entries is a matrix with one column: n × 1.
The rule. To compute entry (i, j) of A @ B, take row i of A and column j of B, multiply the pairs position by position, and add. For that to work, the row and the column must be the same length — which is exactly the shape rule (m × n) @ (n × p) = (m × p). The inner dimensions must match and are consumed; the outer dimensions survive.
Fill in all four entries of A @ B. Click a cell to light up the row of A and column of B it comes from, then reveal the arithmetic.
Shapes: (2 × 3) @ (3 × 2) = (2 × 2). The shared 3 is consumed by the multiply-and-add; the outer 2 and 2 survive.
Every entry is one row laid on one column: multiply the pairs, then add. That is the whole rule — and it is exactly what a neural network layer does, with bigger numbers.
Why not multiply matching cells? Because a matrix is meant to act on a vector, and the action is a weighted sum. Look at A @ [x, y]: row 0 gives 1·x + 2·y and row 1 gives 3·x + 4·y. Each output is a weighted sum of the inputs, with the weights coming from one row. That is precisely what a neuron computes in Lesson 2 — so this is the multiplication that makes matrices useful. Multiplying matching cells is a different, also useful, operation with a different symbol.
Second view: columns combine. The same product reads as a mix of A’s columns, with the input saying how much of each: A @ [x, y] = x · [1, 3] + y · [2, 4]. Numeric check with x = 2, y = 1:
Second worked example, a non-square product. Multiply a 2 × 3 by a 3 × 2:
For A = [[1, 2], [3, 4]] and B = [[5, 6], [7, 8]], what is the top-left entry of A @ B?
Answer before you look. These test the exact moves the rest of the notebook assumes — order of operations, logs, composition, sums and one matrix entry.
0 / 5 answered · 0 correct
01What is 2 + 3 × 4?
02Given ln 2 ≈ 0.693, how else can you write ln 8?
03With g(x) = 3x + 1 and f(u) = u², what is f(g(2))?
04What is Σᵢ₌₁³ i²?
05For A = [[1, 2], [3, 4]] and B = [[5, 6], [7, 8]], what is the top-left entry of A @ B?
Click a card to swap the lazy description for what it actually means.
Four short problems, all doable on paper. Try first; a worked answer is one click away.
Multiplication and division run before addition and subtraction. 2 + 3 × 4: 3 × 4 = 12, then 2 + 12 = 14. (2 + 3) × 4: brackets first, 2 + 3 = 5, then 5 × 4 = 20 — same three numbers, different order, different answer. 10 − 4 ÷ 2: 4 ÷ 2 = 2, then 10 − 2 = 8. Reading left to right would have given (10 − 4) ÷ 2 = 3, which is why the order is a rule and not a suggestion.
ln 8 = ln(2³) = 3 · ln 2 ≈ 3 × 0.693 = 2.079, and e²·⁰⁷⁹ ≈ 8 ✓. For 0.25: 0.25 = 1/4 = 2⁻², so ln 0.25 = −2 · ln 2 ≈ −1.386, and e⁻¹·³⁸⁶ ≈ 0.25 ✓. Numbers below 1 always have negative logs, which is why a probability of exactly 0 gives log(0) = −∞ and breaks code.
For 2i + 1 the index walks 1, 2, 3, 4: terms 2·1+1 = 3, 2·2+1 = 5, 2·3+1 = 7, 2·4+1 = 9, so the sum is 3 + 5 + 7 + 9 = 24. For i²: 1² + 2² + 3² + 4² = 1 + 4 + 9 + 16 = 30. Expand first, add second — the Σ is only shorthand for that instruction.
A @ B: top-left 1·5 + 2·7 = 19; top-right 1·6 + 2·8 = 22; bottom-left 3·5 + 4·7 = 43; bottom-right 3·6 + 4·8 = 50. So A @ B = [[19, 22], [43, 50]]. B @ A: 5·1 + 6·3 = 23; 5·2 + 6·4 = 34; 7·1 + 8·3 = 31; 7·2 + 8·4 = 46, giving [[23, 34], [31, 46]]. Not equal: matrix multiplication is not commutative, so the order you multiply in is part of the meaning.
You do not need to master these here. Each one gets a proper treatment in its own lesson; the one-line meaning is enough to keep reading. Orange dotted underlines in the prose point back to this list.
This lesson is a port of an open course. Everything here traces back to it — and the next step is running the code yourself.
Lesson text adapted from the Math Foundations Notebook's Ground Zero section. The five quiz questions, the worked exercise answers, the second worked examples, and the number-line, function-machine, exponent-log, summation and matrix-trainer labs are original to this page. Every lab runs in your browser.