To effectively probe and break AI systems, you must understand their internal logic. That logic is written in the language of mathematics. This section serves as a practical reference for the core mathematical concepts that form the bedrock of modern machine learning. A firm grasp of these principles will enable you to reason about a model’s vulnerabilities, not just discover them by chance.
Linear Algebra: The Language of Data
At its core, machine learning is about transforming data. Linear algebra provides the structures and operations for representing and manipulating this data efficiently. Nearly all data fed into a neural network—text, images, tabular data—is ultimately converted into numerical structures that linear algebra defines.
Key Data Structures
You will encounter these four fundamental structures constantly:
- Scalar
- A single number. For example, a pixel’s intensity, a price, or a model’s bias term. Represented as x.
- Vector
- An ordered list of numbers. A vector can represent a single data point with multiple features (e.g., a user’s age, income, and location) or a word embedding. Represented as a bold lowercase letter, x.
- Matrix
- A 2D grid of numbers, essentially a collection of vectors. A batch of data, where each row is a data point (vector), is a matrix. The weights of a neural network layer are also stored in a matrix. Represented as a bold uppercase letter, X.
- Tensor
- A generalization of the previous structures to any number of dimensions. A scalar is a 0D tensor, a vector is a 1D tensor, and a matrix is a 2D tensor. A color image (height, width, color channels) is a 3D tensor. Tensors are the standard data structure in deep learning frameworks like TensorFlow and PyTorch.
Figure 1: From a single number (scalar) to multi-dimensional arrays (tensors).
Core Operations
The dot product is a fundamental operation. For two vectors a and b of the same length, their dot product is the sum of the products of their corresponding elements. This is how a neuron computes its initial output: it takes the dot product of the input vector and its weight vector.
Matrix multiplication extends this concept to combine entire layers of neurons, making it the workhorse of deep learning computations.
Calculus: The Engine of Learning
If linear algebra provides the structure, calculus provides the engine for learning. Models learn by minimizing a “loss” or “cost” function, which measures how wrong their predictions are. Calculus allows us to find the minimum of this function efficiently.
Derivatives and Gradients
The key concept is the derivative, which measures the rate of change of a function. For a model’s loss function, the derivative tells us how the loss will change if we slightly adjust a specific weight.
The gradient is simply a vector containing the derivatives for all weights in the model. Critically, the gradient vector points in the direction of the steepest ascent of the loss function. To minimize the loss, we just need to move the weights in the opposite direction of the gradient. This is the core principle of gradient descent, the primary optimization algorithm for training models.
Figure 2: The gradient indicates the “uphill” direction; training involves taking steps “downhill” to minimize loss.
Understanding this process is key to adversarial attacks. Many attacks, like FGSM, work by calculating the gradient of the loss with respect to the input data, not the weights. They then make a small change to the input in the direction that will most increase the loss, causing a misclassification.
Probability & Statistics: Quantifying Uncertainty
Machine learning models are inherently probabilistic. They don’t provide absolute certainties, but rather likelihoods. Understanding the basics of probability is essential for interpreting model outputs and designing robust systems.
Key Concepts
- Probability Distribution
- Describes the likelihood of all possible outcomes. For example, a model classifying images into 10 categories will output a probability distribution across those 10 categories, with the numbers summing to 1.
- Conditional Probability: P(A|B)
- The probability of event A happening, given that event B has already happened. This is the foundation of many models. A language model, for example, calculates the probability of the next word given the sequence of previous words.
- Bayes’ Theorem
- A fundamental formula that relates conditional probabilities. It allows us to update our beliefs in light of new evidence. It’s formally stated as: P(A|B) = [P(B|A) * P(A)] / P(B). This theorem is the basis for entire classes of models, like Naive Bayes classifiers.
For a red teamer, a model’s confidence (the highest probability in its output distribution) is a critical signal. Overconfident incorrect predictions often reveal model brittleness and are prime targets for exploitation.
Common Mathematical Notation
Familiarity with common notation will help you read research papers and technical documentation more easily. Here is a brief reference table.
| Symbol | Name | Meaning in ML Context |
|---|---|---|
| x, y | Lowercase Italic | A scalar value (e.g., a single feature, a single label). |
| x, w, b | Lowercase Bold | A vector (e.g., an input data point, a weight vector, a bias vector). |
| X, W | Uppercase Bold | A matrix (e.g., a batch of input data, a matrix of weights in a layer). |
| θ | Theta | Represents all the parameters of a model (all weights and biases combined). |
| ŷ | y-hat | The predicted output from a model for a given input. |
| Σ | Sigma (Uppercase) | Summation. Used to sum a sequence of numbers, like in the dot product. |
| Π | Pi (Uppercase) | Product. Used to multiply a sequence of numbers. |
| ∇ | Nabla / Del | The gradient operator. ∇J(θ) means the gradient of the loss function J with respect to the parameters θ. |