Project A needs torch 2.4 for CUDA 12.4; Project B needs torch 2.1 because its CUDA build is pinned. Install both globally and one project always loses — that is dependency hell. This lesson gives every project its own environment with uv, venv or conda, records the recipe in pyproject.toml plus a lockfile, and walks through the five mistakes that break it all.
A virtual environment is a directory holding an interpreter link and its own site-packages. Project A's .venv can hold torch 2.4.0 (CUDA 12.4 build) while Project B's holds torch 2.1.0 (CUDA 11.8 build) — same laptop, same day, no argument. A global install is a single slot: the second torch replaces the first, and the project that needed the first fails at import.
one interpreter = one version per packageuv creates environments, installs Python versions and resolves packages; the source's claim is 10–100× faster than pip. Python ships venv, which needs no extra tool and works everywhere. conda earns its place when non-Python dependencies matter — CUDA toolkits, cuDNN, C libraries. Whichever you pick, one rule: never pip-install into a conda environment after conda packages have been installed there.
uv (fast) · venv (built-in) · conda (non-Python deps)pyproject.toml holds the project's requirements, including optional groups like [torch] and [llm], and one command installs them: uv pip install -e ".[torch,llm]". A lockfile (uv.lock) records every exact version — including the transitive ones — so every machine gets the same environment. The .venv directory itself is 200 MB–2 GB, machine-specific, and must stay out of git.
commit pyproject.toml + uv.lock · .gitignore .venv/A virtual environment is a second Python with its own packages: isolation is a box you create, not a setting you switch on — and the things you commit are the recipe (pyproject.toml) and the receipt (uv.lock), never the box itself.
By the end you will be able to explain why two PyTorch versions cannot share one interpreter, create and activate an environment with uv, venv or conda, read a pyproject.toml with [torch] and [llm] extras, install one exact set of packages from a lockfile, check that which python points inside .venv, and diagnose the five mistakes that cause most “it worked yesterday” environment failures — including a CUDA version mismatch between a PyTorch build and the driver.
The source opens with a week in the life of every AI engineer: you install PyTorch 2.4 for a fine-tuning project. Next week a different project needs PyTorch 2.1 because its CUDA build is pinned. You upgrade globally, and the first project breaks. You downgrade, and the second one breaks. That loop has a name, and this lesson exists to end it.
A dependency is any package your code needs in order to run. A version pin is a requirement that says which version — torch==2.4.0 is a pin, torch>=2.3 is a range. Python keeps its packages in one directory per interpreter called site-packages, and that directory can hold exactly one version of any given package name. There is no shelf for “torch 2.4” and another for “torch 2.1”: both are called torch, and installing the second replaces the first. That single fact is dependency hell.
The source names four reasons AI/ML work makes this worse than ordinary Python work — each one is worth reading slowly, because each one is a thing you will actually hit:
cu124 build links against the CUDA 12.4 runtime, the cu118 build against 11.8 — so “install torch” is really “choose a framework version and a CUDA branch”.torch>=2.1,<2.5”), so the framework version stops being a free choice the moment a second library enters the project.pip install overwrites whatever was there before. One interpreter, one slot, no undo — and the overwrite is silent until something imports.Here is the whole disaster in four commands. Read it as a sequence of perfectly reasonable decisions:
# Week 1 — Project A's requirements say torch 2.4.0 (CUDA 12.4 build)
pip install torch==2.4.0
python train.py # fine-tuning works; you move on
# Week 2 — Project B's requirements say torch 2.1.0 (CUDA 11.8 build)
pip install torch==2.1.0 # "installing a dependency", not "changing A"
# Back in Project A — nothing was edited, and it is broken
python train.py
# ModuleNotFoundError: No module named 'torch' (if the name changed)
# or: RuntimeError: CUDA error ... (if the wrong build loaded)
# or worst of all: it runs, with different numbers
# Then the loop: reinstall 2.4, and Project B breaks instead.This is the entire lesson in two lines. Everything that follows —uv, venv, conda, pyproject.toml, lockfiles — is machinery for making “2 interpreters” cheap, reproducible and boring.
The source’s answer to dependency hell is one sentence long: every project gets its own isolated environment with its own packages. No clever resolver, no version gymnastics — a second interpreter with its own site-packages, so two projects stop sharing a single slot.
A virtual environment (the source’s key-terms table calls it “an isolated directory containing a Python interpreter and packages, separate from the system Python”) is exactly that: a directory. It does not copy Python, and it does not duplicate the standard library. On POSIX systems (the Unix-like family — Linux and macOS) .venv/bin/python is a small link to the interpreter that created it, plus a pyvenv.cfg file that tells that interpreter “your site-packages live here, not in the system location”. The packages you install land inside .venv/lib/python3.12/site-packages/. That is a real second slot — which is why torch 2.4.0 and torch 2.1.0 can both exist on one laptop without ever meeting.
Nothing changes until you activate it. Activation is a shell operation: source .venv/bin/activate puts .venv/bin at the front of PATH (the ordered list of directories the shell searches for a program name), so python and pip now resolve inside the box. The prompt changes to (.venv) $ as a receipt, and deactivate puts everything back. Two consequences to keep: activation belongs to the shell, so a new terminal starts unactivated; and an unactivated pip install is the global install from Chapter 01 wearing a normal-looking face.
What isolation is not. A virtual environment is not a sandbox or a security boundary: code in it can still read your files and reach the network. It does not pin anything by itself — it gives each project a place to pin in. It does not make code run faster, and it does not manage Python itself (that is a version manager’s job; on this course’s beginner route, uv python install). It does exactly one job: it makes package versions a per-project decision instead of a per-machine accident.
Two projects, two incompatible PyTorch builds, one machine. The installs happen in order — Project A first, then Project B — and each project’s switch decides whether its install lands in its own .venv or in the shared system interpreter. Flip the switches and watch the conflict appear and disappear.
The model’s one rule is real: an interpreter can hold exactly one version of a package. Isolation is not a feature of PyTorch or of pip — it is a second interpreter with its own site-packages.
You activate Project A's .venv and install numpy 1.26.4. Then you deactivate, activate Project B's .venv, and run `pip show numpy`. What do you see?
All three create the same kind of box. They differ in how much they manage around it: uv also installs Python versions and resolves dependencies quickly, venv ships inside Python and does the minimum, and conda can install the non-Python pieces — CUDA toolkits, C libraries — that a pip wheel assumes are already there.
Start with uv. The source calls it the fastest Python package manager and gives the claim to check against your own machine: 10–100× faster than pip. It is a single binary written in Rust that does three jobs at once — install Python versions, create virtual environments, and install packages — so it replaces the version manager, the environment tool and the installer.
# once per machine
curl -LsSf https://astral.sh/uv/install.sh | sh
# Windows (PowerShell):
# powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"
# a Python to build environments from (no system Python hunting)
uv python install 3.12
cd your-project
uv venv # creates ./.venv with the installed Python
# uv venv --python 3.12 # pin the interpreter explicitly
source .venv/bin/activate # Linux/macOS (.venv\Scripts\activate on Windows)
uv pip install torch numpy # installs into the ACTIVE environment
# example output (illustrative — versions and timings will differ)
# $ uv python install 3.12
# Installed Python 3.12.7 in 1.31s
# + cpython-3.12.7-macos-aarch64-none
#
# $ uv venv
# Creating virtual environment at: .venv
# Activate with: source .venv/bin/activate
#
# $ uv pip install torch numpy
# Resolved 12 packages in 428ms
# Installed 12 packages in 210ms
# + numpy==2.1.3
# + torch==2.4.0For a project with a pyproject.toml, uv has a second style that writes the file for you — this is the one the rest of the lesson builds on:
uv init my-ai-project # creates the folder + pyproject.toml
cd my-ai-project
uv add torch numpy matplotlib
# uv add does three things at once:
# 1. writes the requirement into pyproject.toml
# 2. resolves the full dependency graph
# 3. records every exact version in uv.lock
#
# and this is the line that reproduces the environment later:
uv syncvenv is the fallback. If you cannot install uv — a locked-down work laptop, a cluster whose module system already provides Python — Python ships the same idea in the standard library. It is slower than uv (it still uses pip under the hood), and it works everywhere Python is installed:
python3 -m venv .venv
source .venv/bin/activate # Linux/macOS
.venv\Scripts\activate # Windows
pip install torch numpy # pip, because venv has no resolver of its ownconda earns its keep on the non-Python dependencies. Conda is an environment and package manager that installs compiled libraries too, not just Python wheels — which matters when a framework needs a specific CUDA toolkit, cuDNN (the CUDA Deep Neural Network library of GPU kernels), or a C library without you installing it system-wide. The source names three situations: you need a specific CUDA toolkit version, you are on a shared cluster with no permission to install system packages, or a library’s own instructions say “use conda”.
# Install Miniconda (minimal), not the full Anaconda distribution
curl -LsSf https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -o miniconda.sh
bash miniconda.sh -b
conda create -n myproject python=3.12
conda activate myproject
# the packages come from conda channels, not from PyPI
conda install pytorch torchvision torchaudio pytorch-cuda=12.4 -c pytorch -c nvidia| tool | manages | pick it when | first command |
|---|---|---|---|
uv | Python versions, environments, packages, lockfiles | almost always — it is the course default | uv venv |
venv | environments only (pip does the installing) | you cannot install uv, or Python already exists and is managed | python3 -m venv .venv |
conda | Python + non-Python libraries (CUDA toolkits, C libraries) | a non-Python dependency must be installed without touching the system | conda create -n myproject python=3.12 |
Activation is not a mode of the project — it is a state of the shell. Type the four commands and watch which python and which pip change what they answer, and where the same pip install torch ends up. Nothing here touches a real machine; the paths are the ones you would see in a real project.
Nothing is broken — yet. The install works, it just works in the wrong place, and every project on this machine now sees it.
A library's install page says it needs the CUDA 12.4 toolkit and cuDNN present as system libraries, and you are on a shared cluster where you cannot install system packages. Which tool matches the source's advice?
You could install everything this course will ever need into one environment and never think about it again. The source gives that idea two sentences and one verdict: different phases need different, and sometimes conflicting, dependencies. So the environment is a per-phase decision, made when the phase is about to start.
Two acronyms to have ready before the tree: API (application programming interface — the hosted model service a program sends requests to) and SDK (software development kit — the package that wraps that service in code), which together name the LLM (large language model) work that needs no GPU at all. The source draws the strategy as a directory tree. Read it as a map of which work can share a box and which cannot: the early phases are light and friendly, the neural-network phases are heavy, and the transformers phase may disagree with the neural-network phase about which transformer versions (and which torch line) it wants.
ai-engineering-from-scratch/
├── .venv/ <-- shared lightweight env for phases 0-3
├── phases/
│ ├── 04-neural-networks/
│ │ └── .venv/ <-- PyTorch env
│ ├── 05-cnns/
│ │ └── .venv/ <-- same PyTorch env (symlink or shared)
│ ├── 08-transformers/
│ │ └── .venv/ <-- might need different transformer versions
│ └── 11-llm-apis/
│ └── .venv/ <-- API SDKs, no torch neededOne honesty note about the folder names: the source’s tree is a sketch, and upstream’s current curriculum does not have a phases/08-transformers/ directory — its transformer material lives across the Transformers Deep Dive and Generative AI phases, and its LLM API work in LLM Engineering. Read the tree for its roles, not its literal paths: a light base for the early phases, a PyTorch environment for the training phases, a transformers environment that may disagree with it, and a small API-only environment.
Three habits make the tree work in practice. First, create an environment when a lesson asks for one, not on day zero — the source’s own philosophy from Lesson 01, and the reason the early phases start quickly. Second, name it after the project or phase (phases/04-neural-networks/.venv), not venv or myenv: three terminals and two months later, the name is the only documentation left. Third, let phases share when they genuinely agree — Phase 05 uses Phase 04’s PyTorch environment, per the source’s comment — and split the moment a version disagrees. The environment boundary should follow the dependency boundary, not the folder structure.
The course ships a script for the shared base environment — code/env_setup.sh — which is also a good template for your own setup scripts. It checks for uv and falls back to python3 -m venv, requires Python 3.11+, creates .venv at the repository root, installs numpy matplotlib jupyter scikit-learn pandas, and then verifies every import, prints each package’s version, runs a small NumPy matrix multiply, and reports PyTorch as “warn” rather than failure if it is not installed yet.
bash phases/00-setup-and-tooling/06-python-environments/code/env_setup.sh
# what it does, in order:
# 1. finds uv, or falls back to python3 -m venv
# 2. checks Python >= 3.11 and creates/reuses .venv
# 3. activates it and proves python runs FROM the venv
# 4. installs numpy matplotlib jupyter scikit-learn pandas
# 5. verifies each import and prints version numbers
# 6. reports PyTorch as "install later when needed" if it is absent
# the same idea, typed by hand:
uv python install 3.12
uv venv
source .venv/bin/activate
uv pip install numpy matplotlib jupyter scikit-learn pandasWhere the single environment looks cheaper, it is only because it has not hit its conflict yet. The moment Phase 08 needs a different torch than Phase 04, disk is no longer the interesting number: the one environment is simply wrong, and the only fix is the split you avoided. The source’s own qualification is worth keeping: phases that agree can share (04 and 05 do), so the goal is not “one environment per chapter” — it is “one environment per dependency agreement”.
The source says it plainly: do not build one environment for the whole course. Pick a strategy and the phase groups you plan to work through, and see the environments you would end up maintaining — plus the problems one shared interpreter cannot avoid. The phase labels follow the source’s sketch (upstream’s folder names differ — read the roles). Sizes are teaching estimates: the source’s typical range is 200 MB–2 GB per environment, and the API-only box is deliberately lighter because no compiled stack ships.
The honest reading: one environment is not “free” — it is cheap until the first conflict, and then it is the most expensive thing on the machine. Per-phase environments cost disk; a broken single environment costs the afternoon you spend untangling it.
The source is blunt about it: every Python project should have a pyproject.toml, and it replaces setup.py, setup.cfg and requirements.txt in one file. One file describes what the project needs; the environment is built to match it, and the lockfile (Chapter 06) records what that build produced.
pyproject.toml is a TOML file — Tom’s Obvious Minimal Language, a configuration format designed to be readable without a parser in your head — and its [project] table is standardised by PEP 621 (a Python Enhancement Proposal, the community’s standards process). You do not need the standards to read it; you need to know that this file, not the environment, is the source of truth. Here is the course’s own file, verbatim from the source:
[project]
name = "ai-engineering-from-scratch"
version = "0.1.0"
requires-python = ">=3.11"
dependencies = [
"numpy>=1.26",
"matplotlib>=3.8",
"jupyter>=1.0",
"scikit-learn>=1.4",
]
[project.optional-dependencies]
torch = ["torch>=2.3", "torchvision>=0.18"]
llm = ["anthropic>=0.39", "openai>=1.50"]Four things in that file deserve a sentence each. name and version identify the project — the version is the project’s own, unrelated to any package’s. requires-python = ">=3.11" is the interpreter floor; it is the same 3.11+ check the course’s env_setup.sh enforces, while the environments this lesson builds use 3.12. dependencies is the set every user of the project gets, and optional-dependencies is where AI projects keep their heavy or situational stacks: torch for the phases that train models, llm for the phases that call APIs. An extra is a named group; installing one means installing the base plus that group. Extras are >= ranges, not exact pins — the lockfile’s job, coming in the next chapter.
The source then shows the three install lines that follow from the file:
uv pip install -e ".[torch]" # base + PyTorch
uv pip install -e ".[llm]" # base + LLM SDKs (small, API-only)
uv pip install -e ".[torch,llm]" # everything
# the -e . at the start installs the project directory itself in editable
# mode, so imports resolve to your working copy.
# the [bracketed] names select extras from [project.optional-dependencies].
#
# uv's project-style equivalent (writes uv.lock for you):
uv sync --extra torch --extra llmThe payoff of extras is that one file serves several audiences without lying to any of them. A CI job that only runs the API tests can install .[llm] and spend seconds, not gigabytes — the dependency-tree lab below shows the difference in nodes. A GPU machine installs .[torch] and gets the CUDA runtime wheels with it (on Linux and Windows; macOS gets the Metal build). A reader who wants everything asks for both.
The project’s pyproject.toml lists four base packages and two optional groups. Switch the groups on and watch the resolver pull in the packages those groups need — including the GPU wheels that make a torch install heavy. The tree is an illustrative subset: the real resolver returns a larger graph, and jupyter alone expands into dozens of packages collapsed here into one node.
Each package names every requirement that pulled it in (via); a package with two via links is shared, which is exactly why one environment can hold it once.
jupyterbasematplotlibbasenumpybasebasebasescikit-learnbasecontourpybasecyclerbasefonttoolsbaseipykernelbaseipywidgetsbasejoblibbasejupyterlabbasekiwisolverbasenbconvertbasenotebookbasepackagingbasepillowbasepyparsingbasepython-dateutilbasescipybasesixbasethreadpoolctlbaseWithout [torch], this tree is small: the base plus whichever extras you chose. Nothing here needs a GPU.
Read the counts as a lesson in economics: the base group is dozens of packages; the [llm] group is a handful of small SDK packages; the [torch] group is the one that drags in a GPU runtime as ten separate wheels. That is the whole reason phases can share one lightweight environment and still need separate heavy ones.
Your pyproject.toml has torch = ["torch>=2.3", "torchvision>=0.18"] and llm = ["anthropic>=0.39", "openai>=1.50"]. Which command installs the base dependencies plus the two LLM SDKs, and no torch at all?
>= is a wish.A lockfile pins every dependency — including the transitive ones you never named — to exact versions, so anyone who installs from it gets exactly the same packages. The source’s instruction is one line: commit your lockfile to git. This chapter is why that line matters.
Chapter 05’s pyproject.toml says numpy>=1.26 and torch>=2.3. That is a range, and a range is a promise to be flexible — which is exactly wrong for reproducibility. Two machines installing the same file on different days get different versions, because “newest that satisfies >=” is a moving target. A transitive dependency (the key-terms table puts it plainly: a dependency of a dependency) makes this worse, because the packages that actually break your code are usually ones you never wrote down: you asked for scikit-learn, and scipy and joblib arrived. The lockfile is the record of what the resolver chose, down to the last transitive package.
# uv's project workflow: uv.lock appears and is maintained automatically
uv add numpy
# the pip-tools-shaped workflow, also built into uv:
uv pip compile pyproject.toml -o requirements.lock
uv pip install -r requirements.lock
# what the lock adds (illustrative shape of the file — not a real excerpt)
# numpy==1.26.4
# torch==2.4.0
# transformers==4.44.2
# ... every transitive package, exact, with platform markers
# and the one habit that makes it count:
git add pyproject.toml requirements.lock # or uv.lock
echo ".venv/" >> .gitignoreTwo properties are worth spelling out because they explain the lab. First, exactness: a lockfile is a list of name==version pins, including transitive packages — so the four direct base dependencies in the toy project expand to the 21 nodes the dependency-tree lab showed, and all 21 are pinned. Second, portability of the recipe: an environment is local and machine-specific, but the lock is text, so it travels through git, code review and CI. A Mac and a Linux CI runner can both install from one uv.lock and each pick the right wheels, because the lock records the resolution together with platform markers. The partition of responsibilities is clean: pyproject.toml is what you want, the lock is what you got, and .venv is where it landed — commit the first two, ignore the third.
Three machines install the same project on three different dates: the author’s laptop, a teammate’s Mac six months later, and the continuous-integration (CI) runner today. With a lockfile every machine gets the identical set; with loose minimum versions each machine gets whatever was newest that day. The versions and dates are illustrative — the drift pattern is the real thing.
The lockfile does not make the install faster or smaller; it makes it the same. A min/max constraint like numpy>=1.26 is a wish, and every machine grants it differently; the lock spreads one granted wish across the team.
Notice what the lockfile does not do: it does not make the install faster, does not shrink the environment, and does not stop you from upgrading. It converts “works on my machine” from a complaint into a test, because the CI runner is now provably installing the same environment the author used. When you do want newer packages, the upgrade is one deliberate act — uv lock --upgrade (or uv add package --upgrade) — followed by a lockfile change in a diff someone can review, instead of a silent accident on whichever machine installed last.
The source closes the build with a list of five mistakes, each with the exact command that causes it and the exact check that proves you are clear. Read them as a preflight: run the checks now, once, and most of the “it worked yesterday” failures never happen to you.
A global pip install writes into the system Python’s site-packages — the single shared slot from Chapter 01. The fix is one line of activation, and the proof is two which commands:
pip install torch # BAD: installs to the system Python
source .venv/bin/activate
pip install torch # GOOD: installs to the virtual environment
# check where your packages actually go:
which python # should show .venv/bin/python, not /usr/bin/python
which pip # should show .venv/bin/pip
# the failsafe that never needs activation to be right:
python -m pip install torch # "the pip that belongs to THIS python, please"Conda solves an environment as one graph; pip installs straight past the solver. The source’s rule is absolute inside a conda environment — use conda for all packages. If a package is pip-only, install all conda packages first and the pip packages last, and never upgrade a conda-managed package with pip.
conda create -n myenv python=3.12
conda activate myenv
conda install pytorch -c pytorch
pip install some-other-package # BAD: can break conda's dependency tracking
conda install some-other-package # GOOD: let conda manage everything
# if you truly must mix: conda packages first, pip packages last,
# then check which packages came from PyPI:
conda list | grep pypiThe environment exists, the packages are installed, and the commands still fail — because the shell is pointing at the system Python. The prompt and which are the two tells.
python train.py # uses system Python, missing packages
# ModuleNotFoundError: No module named 'torch'
source .venv/bin/activate
python train.py # uses project Python, packages found
# your shell prompt should show the environment name:
(.venv) $ python train.py
# and this must print a path inside .venv:
python -c "import sys; print(sys.executable)"An environment is 200 MB–2 GB of machine-specific binaries with absolute paths baked in (remember the anatomy diagram — the interpreter is a link, and links point at one machine’s disk). It is not portable, it bloats the repository forever, and it is regenerated in minutes from the recipe. Ignore it, and commit the two files that recreate it.
echo ".venv/" >> .gitignore
# commit these instead:
# pyproject.toml the recipe (what the project wants)
# uv.lock the receipt (exact versions that satisfied it)
# env_setup.sh a script that rebuilds a shared base environment
# if it is already committed, untrack it without deleting your copy:
git rm -r --cached .venv
echo ".venv/" >> .gitignore
git commit -m "stop tracking the virtual environment"The one mistake from this lesson that is not about Python. Two commands report two different CUDA versions, and they have an order. nvidia-smi asks the driver what it supports; torch.version.cuda reports what the installed wheel was compiled against. The rule from the source: the PyTorch CUDA version must be ≤ the driver’s CUDA version.
nvidia-smi
# Driver Version: 550.54.14
# CUDA Version: 12.4 <-- the driver's ceiling (illustrative output)
python -c "import torch; print(torch.version.cuda)"
# 12.4 <-- what this torch build was compiled for
# These must be compatible:
# PyTorch CUDA version must be <= driver CUDA version.
# a cu121 build on a 12.4 driver → fine (the driver is newer)
# a cu124 build on a 12.1 driver → CUDA not available
python -c "import torch; print(torch.cuda.is_available())"
# True <-- the only question that mattersRead the driver’s CUDA version from nvidia-smi and PyTorch’s from torch.version.cuda. The wheel may be built for the same CUDA version as the driver or older — never newer. Pick a pair and watch the rule decide. The matrix is the source’s rule, not a driver download table; real drivers also have a minimum version per CUDA branch (listed below).
| driver / torch | ||||
|---|---|---|---|---|
torch cu124 needs exactly CUDA 12.4; the driver reports 12.4. Exact matches never argue.
The full rule, in the source’s words: torch’s CUDA version must be ≤ the driver’s CUDA version. Real drivers also carry a floor per branch — CUDA 11.x builds: ≥ 450.80.02 · CUDA 12.x builds: ≥ 525.60.13 — so a machine old enough to lack those drivers fails for a second reason as well.
Two failure modes look identical and have different fixes: a wheel built for a newer CUDA than the driver gives torch.cuda.is_available() == False, while a missing NVIDIA GPU on the machine gives the same False. The matrix distinguishes them.
You run `pip install torch` and it reports success. A moment later `python train.py` fails with `ModuleNotFoundError: No module named 'torch'`. What is the most likely explanation?
Answer before you look. The lockfile question and the CUDA question are the two that separate “I read the lesson” from “I can set up a project that still works next month”.
0 / 6 answered · 0 correct
01What problem do virtual environments solve?
02What is a lockfile in the context of Python dependency management?
03How can you verify that your pip and python commands are using the virtual environment and not the system Python?
04Why is mixing pip and conda in the same environment problematic?
05Your PyTorch code reports 'CUDA not available' despite having an NVIDIA GPU. What is the most likely cause?
06You cloned a project that ships pyproject.toml and a committed uv.lock. A teammate installed from the loose `>=` requirements instead of the locked path, and two weeks later their tests fail while yours pass. What is the most likely difference?
Click a card to swap the lazy description for what it actually means.
Four small drills — run the setup script and read its checks, prove two environments are isolated, write a project file for PyTorch plus the Anthropic SDK, and deliberately install something globally so you can see where it goes. Try first; a worked answer is one click away.
From the repository root, run `bash phases/00-setup-and-tooling/06-python-environments/code/env_setup.sh`. Read the output as a sequence of claims being tested, not decoration. (1) `[PASS] uv found: uv 0.x.y` (or `[WARN] uv not found`) — the script will fall back to `python3 -m venv` + pip, so a missing uv is a warning, not a failure. (2) A Python probe with `PYTHON_MIN_MAJOR=3`, `PYTHON_MIN_MINOR=11`: anything older prints `[FAIL] Python 3.11+ not found` with the three install suggestions (`uv python install 3.12`, `brew install python@3.12`, `sudo apt install python3.12 python3.12-venv`). (3) `[PASS] Created .venv` or `[WARN] Existing .venv found. Reusing it.` — the script is idempotent; rerunning it does not destroy your environment. (4) The two checks that catch the lesson's mistakes: `[PASS] Activated virtual environment`, then the path assertion — if `which python` does not contain `.venv`, the script exits with `[FAIL] Python is not running from the venv: <path>`. (5) Installation of `numpy matplotlib jupyter scikit-learn pandas`, then one verification line per package that prints its real `__version__` (for example `numpy: 2.1.3`), plus a NumPy matrix-multiply smoke test printing `Matrix multiply check: (3, 3) @ (3, 3) = (3, 3)`. (6) A PyTorch probe that is intentionally a warning: `[WARN] PyTorch not installed (install later when needed)` with the `uv pip install torch torchvision torchaudio` line — the Lesson 01 philosophy that later tools are installed when a lesson asks for them. (7) `[PASS] All checks passed` and the activation reminder `source <repo>/.venv/bin/activate`. The only genuine failure exit is a FAILURES count above zero (missing/too-old Python, an unextractable activation script, or a failed package verification); arbitrary version numbers in the PASS lines are fine — the script is testing presence and importability, not a specific version.
Build the second box next to the first, then prove each interpreter sees only its own packages. With uv: `uv venv .venv-b`, `source .venv-b/bin/activate`, `uv pip install "numpy==1.26.4"`, then `python -c "import numpy; print(numpy.__version__, numpy.__file__)"` prints `1.26.4` and a path inside `.venv-b/lib/python3.12/site-packages/`. Now `deactivate`, `source .venv/bin/activate`, and the same command prints whatever the first environment has (for a fresh course environment from `env_setup.sh`, the newest numpy the resolver chose — say `2.1.3`) with a path inside `.venv/lib/...`. Two checks worth running while you are there: `which python` must print the path of the environment you just activated, and `pip show numpy` in the environment that never installed numpy should report `Package(s) not found` — not the other box's copy. That absence is the proof of isolation; nothing is shared, so nothing can be overwritten. The same drill with the fallback tool is `python3 -m venv .venv-b` and then `pip install "numpy==1.26.4"` (no `uv` prefix) — same result, slower install. Clean-up note: `.venv-b` is disposable (`rm -rf .venv-b`); this is why environments are not committed and why the two files that recreate them are.
Keep the two heavy stacks in extras so each audience installs only what it needs: `[project]` with `name = "my-ai-project"`, `version = "0.1.0"`, `requires-python = ">=3.11"`, and `dependencies = ["numpy>=1.26", "matplotlib>=3.8"]`; then `[project.optional-dependencies]` with `torch = ["torch>=2.3", "torchvision>=0.18"]` and `llm = ["anthropic>=0.39", "openai>=1.50"]`. The install lines mirror the source: `uv pip install -e ".[torch]"` for the training stack, `uv pip install -e ".[llm]"` for the API work, and `uv pip install -e ".[torch,llm]"` for both; with uv's project workflow the equivalent is `uv add torch torchvision` and `uv add anthropic openai`, then `uv sync --extra torch --extra llm` on any machine that clones the project. Three design notes make the file good rather than merely valid. First, the Anthropic SDK belongs in an extra, not in `dependencies` — it is small, but not everyone using the project needs it. Second, the ranges are deliberately loose (`>=`) because `uv.lock` is where exactness lives; writing `torch==2.4.0` in the project file makes every machine's resolution a manual approval. Third, `requires-python` should match reality: the course's own environments are built from Python 3.12, and the setup script refuses anything below 3.11.
Start a fresh terminal so no environment is active, and check where the tools point before installing anything: `which python`, `which pip`, and `python3 -m pip --version` (which prints the interpreter pip is attached to — the check that catches the tangle behind mistake 3). Now try a deliberately tiny package: `python3 -m pip install six`. On many modern systems the install is refused before it starts, with a message about an `externally-managed-environment`; that is PEP 668 (Marking Python base environments as 'externally managed') working as designed, protecting the operating system's Python from exactly this lesson's global-install mistake. If it is refused, you have still learned the lesson — and you should not reach for `--break-system-packages`. If it installs, find out where it landed with `python3 -c "import six; print(six.__file__)"`: a system path like `/usr/lib/python3/dist-packages/six.py`, or a per-user path under `~/.local/lib/python3.12/site-packages/` if pip used `--user` mode. Either way, note the shape of the result: the package is now visible to every script on the machine that runs this interpreter. Uninstall it with `python3 -m pip uninstall -y six` and confirm `python3 -c "import six"` fails again. Now do the contrast experiment: create and activate an environment (`uv venv .venv-lab`, `source .venv-lab/bin/activate`), install the same package (`uv pip install six`), and print `six.__file__` — the path is inside `.venv-lab`, and `deactivate` makes it invisible to the system Python again. That pair of paths is the entire difference between mistake 1 and correct practice, demonstrated on your own machine.
You do not need to master these here. Each one gets a proper treatment in its own lesson; the one-line meaning is enough to keep reading. Orange dotted underlines in the prose point back to this list.
This lesson is a port of an open course. Everything here traces back to it — and the next step is running the code yourself.
Lesson text, quiz and the env_setup.sh walkthrough are adapted from AI Engineering from Scratch (Phase 00, Lesson 06). Everything the source states is kept as-is: the two-project PyTorch 2.4/2.1 story with its global-install overwrite, the four reasons AI makes dependency hell worse, the three tools and their exact commands (uv's 10–100× claim, `python3 -m venv .venv`, miniconda plus `conda create -n myproject python=3.12`), the one conda rule about pip installs, the per-phase course tree, the pyproject.toml with [torch] and [llm] extras and its three install lines, the lockfile commands with "commit your lockfile", the five common mistakes with their checks, and the source's exercises. Original to this page: the five labs (the canvas isolation simulator, the dependency-tree board, the canvas activation state machine, the CUDA compatibility matrix and the lockfile reproducibility lab) plus the course environment planner; the slot arithmetic (2 interpreters × 2 requirements vs 1 × 2 → 1 broken project); the anatomy-of-.venv figure and the "what isolation is not" paragraph; the uv project workflow, the Windows installer line and the uv pip install vs uv add distinction; the per-phase cost estimate (4 environments ≈ 3.2–5.6 GB versus 1 ≈ 0.2–2 GB, teaching estimates inside the source's 200 MB–2 GB range) with the note that upstream's folder names differ from the source's sketch; the field-by-field pyproject reading with TOML and PEP 621 spelled out; the environment-drift repair habit; the three-machine lockfile drift table (numpy 1.26.4 → 2.3.5, torch 2.4.0 → 2.8.0, transformers 4.44.2 → 4.55.0, one failing install — illustrative versions and dates); the memory hooks "a .venv is a box, not a copy", "the wish and the receipt" and "the driver is the ceiling"; the fifth mistake's extended checks (python -m pip, sys.executable, conda list | grep pypi, git rm -r --cached .venv); and the sixth quiz question about loose requirements versus a committed lock. Version numbers, dates, package counts, disk sizes and example command outputs are labeled teaching estimates; the labs compute their numbers live, and every canvas is a simulation rather than a benchmark.