Setting Python Environment

Auditing an ML repository follows a systematic inspection checklist:

1. Root Configuration Inspection:

• `README.md`: Identifies hardware recommendations (GPU compute capability, VRAM size), software prerequisites, and baseline checkpoints.
• `pyproject.toml` or `setup.py`: Identifies project build-backends, package metadata, and entry points.
• `requirements.txt`: Maps exact package versions and potential dependency conflicts.

1. Execution Vector Mapping:

• Determine whether the project is designed as an importable library (`import llms_from_scratch`) or a command-line script pipeline (`python train.py --config config.yaml`).

1. Hardware & Resource Requirements:

• Compute footprint: CPU-only vs. NVIDIA CUDA vs. Apple Silicon MPS.
• Disk storage: Datasets (10–500 GB) and model checkpoints (GPT-2 124M weights require ≈ 500 MB; 7B weights require ≈ 14 GB).

In enterprise AI labs, executing untrusted code without repository audits violates security compliance. Engineers verify cryptographic commit signatures, inspect sub-dependencies for malicious supply-chain packages, and run sandboxed containerized evaluations before provisioning GPU cluster resources.

In the early days of deep learning (Caffe and Theano era, 2010–2014), setting up a project often required manual compiling of CUDA drivers and editing Makefiles for days. Modern Python packaging standards (PEP 517/518) streamlined builds, but the rapid evolution of PyTorch and CUDA toolkits makes methodical upfront inspection essential.

• Read before you run; inspect before you install.
• A five-minute audit of the README prevents a five-hour fight with CUDA drivers.

A standard educational and production LLM repository adheres to clear structural conventions:

```text

llms-from-scratch/

├── ch01/ # Overview and setup scripts

├── ch02/ # Data preprocessing, tokenization, DataLoader

├── ch03/ # Multi-head self-attention mechanisms

├── ch04/ # Complete GPT-2 model architecture

├── ch05/ # Pretraining loop, evaluation loss, generation

├── ch06/ # Text classification fine-tuning

├── ch07/ # Instruction fine-tuning (SFT)

├── requirements.txt # Pinned dependency specifications

└── README.md # Setup and execution guide

```

Tracing code dependencies flows sequentially:

ch02 (Data) →(Weights) ch05 (Pretraining)

Understanding this modular pipeline prevents importing code out of order.

Production codebases like Megatron-LM or Hugging Face Transformers separate dataset streaming, distributed communication primitives (`torch.distributed`), and model forward passes into separate modular subpackages. Navigating this structure quickly is a core skill for ML infrastructure engineers.

Sebastian Raschka structured the LLMs-from-scratch open-source repository as an organic pedagogical progression. Rather than dumping an opaque 5,000-line library, each chapter builds incrementally on the previous one, allowing students to verify each subsystem in isolation.

• Map the repository layout before firing the execution engine.
• Data feeds attention; attention feeds the transformer; the transformer feeds the training loop.

Deterministic execution requires stability across three distinct layers:

1. Layer 1: System & Hardware Layer

• Host Operating System (Linux x86_64, macOS ARM64).
• NVIDIA Display Driver & CUDA Kernel Driver (queried via `nvidia-smi`).
• Hardware Accelerators (GPU VRAM, Apple Unified Memory).

1. Layer 2: Virtual Environment Layer

• CPython runtime binary (`bin/python` targeting e.g. 3.10.12).
• Pinned Python package wheels in `site-packages/` (`torch==2.3.0`, `tiktoken==0.7.0`).
• Isolated `PATH` and `PYTHONPATH` environment variables.

1. Layer 3: Application Code & Data Layer

• Repository source files (`.py`, `.ipynb`).
• Hyperparameter configurations and random seeds (`torch.manual_seed(123)`).
• Tokenizer vocabularies and dataset binary caches.

Reproducibility = Layer 1 (Drivers) ∧ Layer 2 (Environment) ∧ Layer 3 (Code & Seeds)

In enterprise machine learning operations (MLOps), Docker containers and Kubernetes clusters freeze Layers 1 and 2 into immutable container images (`nvcr.io/nvidia/pytorch:24.03-py3`). This guarantees that training runs on an 8-GPU node reproduce identical loss curves on a 1,000-GPU cluster.

The 'it works on my machine' crisis plagued early data science. The introduction of virtual environments in PEP 405 (Python 3.3, 2011) and Docker containerization (2013) transformed experimental code into portable, industrial-grade software engineering pipelines.

• System drivers underneath; virtual environment in the middle; application code on top.
• If any layer shifts, reproducibility breaks.

Semantic versioning follows the convention `MAJOR.MINOR.PATCH`:

• MAJOR: Breaking API changes (e.g., PyTorch 1.x → 2.0 introduced `torch.compile`).
• MINOR: New functionality without breaking backward compatibility (e.g., PyTorch 2.2 → 2.3).
• PATCH: Backward-compatible bug fixes and performance patches.

In deep learning, however, even patch and minor releases can change:

1. Default random weight initialization distributions.
2. Tensor casting rules between BF16 and FP32.
3. Underlying cuDNN convolution or attention algorithm selections.

To synchronize:

Specification: \texttttorch>=2.2.0, <2.4.0 ⟹ Pip Resolver selects highest compatible 2.3.x

Production teams maintain `requirements.lock` or `Pipfile.lock` containing exact sha256 wheel hashes. When a model trains for 3 weeks on 1024 H100 GPUs, an unexpected sub-dependency update in the middle of a node restart could corrupt the checkpoint resumption.

The transition from PyTorch 1.x to 2.0 in March 2023 introduced TorchDynamo and Inductor graph compilation. Codebases that didn't pin versions experienced immediate compilation errors or unexpected tensor graph capture issues.

• Lock your versions, or the versions will unlock chaos.
• Reproducibility begins with identical package versions.

Comparison of standard Python environment pathways:

1. Standard `venv` (Recommended for pure PyTorch):

• Built into Python standard library (`python3 -m venv venv`).
• Lightweight, zero system modifications.
• Relies directly on PyTorch's official pre-compiled wheels containing embedded CUDA runtime libraries.

1. Conda / Mamba:

• Manages non-Python system dependencies (C++ compilers, BLAS, CUDA Toolkit).
• Higher disk footprint (2–5 GB baseline).
• Strict channel precedence rules (`conda-forge` vs `defaults` vs `pytorch`).

1. `uv` (Modern Rust-based toolchain):

• Extremely fast resolver and environment manager (10–100× faster than pip).
• Fully compliant with standard Python wheel specifications.

Decision Rule:

Stick strictly to: \textttpython3 -m venv .venv ⟹ \textttsource .venv/bin/activate ⟹ \textttpip install

Machine learning teams standardize on a single organization-wide toolchain. A standard setup is Python's native `venv` orchestrated inside minimal Ubuntu Docker images, avoiding heavy Conda layers in production deployments.

Conda was created by Continuum Analytics in 2012 because pip could not manage compiled scientific C/Fortran libraries like NumPy and SciPy on Windows and macOS. With the arrival of the binary wheel format (PEP 427, 2012), pip became capable of installing complex pre-compiled binaries, reducing the need for Conda in modern PyTorch projects.

• Pick one environment path and walk it faithfully.
• Mixing package managers creates binary graveyards.

When a command is entered, the shell searches directories defined in `PATH`:

PATH = dir₁ : dir₂ : dir₃ : ṡ

When you activate a virtual environment (`source .venv/bin/activate`):

PATH arrow /path/to/project/.venv/bin : PATH

Verification commands:

```bash

which python3

# Correct output: /Users/username/.../project/.venv/bin/python3

python3 -c "import sys; print(sys.executable)"

# Output matches the active virtual environment binary

```

If `which python3` prints `/usr/bin/python3` or `/opt/homebrew/bin/python3`, your virtual environment is NOT active.

Automated CI/CD pipelines (GitHub Actions, GitLab CI) never rely on `source activate`. Instead, they explicitly invoke the virtual environment binary directly:

`/path/to/.venv/bin/python -m pytest tests/`

This eliminates any possibility of `PATH` ambiguity across execution agents.

The `PATH` environment variable was introduced in Unix Version 7 in 1979 by Stephen Bourne. Decades later, it remains the universal mechanism for process execution across macOS, Linux, and POSIX-compliant operating systems.

• Trust `which python3`, not your assumptions.
• The leftmost directory in `PATH` always wins the execution race.

Operating systems (especially Ubuntu, Debian, macOS) utilize system Python for critical internal tasks:

• `apt` package manager integration (`python3-apt`).
• System daemon services, printing stacks, and networking scripts.

Executing `sudo pip install torch` overwrites shared C-libraries in `/usr/lib/python3/dist-packages/`, leading to:

1. Dependency Deadlocks: System packages expecting `urllib3<2.0` breaking when an ML tool installs `urllib3==2.2`.
2. C++ ABI Mismatches: Clashing versions of `glibc`, `libgomp`, or OpenMP runtime symbols.
3. PEP 668 Protection: Modern Linux distros deliberately block global pip installs with `externally-managed-environment` errors to protect OS integrity.

An isolated environment decouples project dependencies entirely:

OS Utilities → System Python \bot LLM Code → Project .venv

Every serious AI research lab mandates containerization or isolated virtual environments. Engineers working on multiple models simultaneously (e.g. PyTorch 2.1 for an older LLaMA-2 repo, and PyTorch 2.4 for a new Llama-3 repo) switch between clean `.venv` folders without conflict.

PEP 668 ('Marking Python base environments as \"externally managed\"') was adopted in Python 3.11. It officially locked down system Python installations across Debian, Ubuntu, and Homebrew, forcing developers to adopt virtual environments as standard practice.

• Never touch system Python; always build inside a sandbox.
• Containment protects your OS; isolation protects your experiment.

Understanding PEP 440 comparison operators:

• `torch == 2.2.2`: Exact pin (deterministic).
• `torch >= 2.2.0, < 2.4.0`: Bounded range (allows non-breaking minor/patch updates).
• `torch ~= 2.2.0`: Compatible release clause (equivalent to `>= 2.2.0, == 2.2.*`).
• `torch`: Unbounded (unsafe; installs latest release regardless of compatibility).

The pip backtracking resolver constructs a dependency constraint graph:

G = (V, E), V = packages, E = version constraints

If Package A requires `numpy<1.24` and Package B requires `numpy>=1.26`, the resolver halts with an unsatisfiable conflict error (`ResolutionImpossible`).

Production LLM teams inspect dependencies for compiled binary wheel availability before initiating cluster jobs. If a dependency lacks a pre-compiled wheel for Python 3.12 on Linux ARM64, pip will attempt to build from source, triggering missing C++ header errors.

In 2020, pip 20.3 rolled out a brand-new backtracking dependency resolver. Previously, pip used a naive greedy first-match approach that frequently installed silently broken combinations of packages.

• Unpinned dependencies are bugs waiting to explode.
• Exact pins (`==`) yield reproducibility; loose pins (`>=`) invite drift.

A robust verification script tests three fundamental capabilities:

```python

import torch, tiktoken

# 1. Verify library versions

print("PyTorch version:", torch.__version__)

print("Tiktoken version:", tiktoken.__version__)

# 2. Verify Hardware Acceleration

device = "cuda" if torch.cuda.is_available() else (

"mps" if torch.backends.mps.is_available() else "cpu"

)

print("Active compute device:", device)

# 3. Verify Tensor Arithmetic on Device

x = torch.tensor([1.0, 2.0, 3.0], device=device)

y = x * 2.0

assert y.sum().item() == 12.0

print("Hardware tensor operations verified successfully!")

```

This smoke test verifies:

• Dynamic library linkage (`libtorch.so`, `libc10.so`).
• GPU driver context creation.
• Accurate floating-point tensor execution on the targeted device.

Cluster orchestration engines (Slurm, Ray, Kubernetes) run this exact smoke test on every compute node as a pre-flight health check before dispatching multi-node distributed training jobs.

Hardware acceleration in Python evolved from manual CUDA C extensions in the late 2000s to PyTorch's unified `torch.device` abstraction introduced in PyTorch 0.4 (2018), allowing seamless device-agnostic tensor programming.

• Never trust a silent install; always run the smoke test.
• Allocate a tensor on device before allocating hours to training.

The distinction between `!` and `%` in IPython:

1. Shell Escape (`!pip install`):

• Spawns a temporary subshell: `/bin/sh -c "pip install ..."`.
• Uses the shell's default `PATH`.
• If the Jupyter server was launched from a different environment, `pip` resolves to the host or system Python, not the running kernel's environment.

1. IPython Magic (`%pip install`):

• Directly queries the running kernel's executable:

\texttt\%pip install ... ≡ \textttsys.executable -m pip install ...

• Guarantees that packages land in the exact `site-packages` directory of the active kernel.

In Google Colab, Kaggle Kernels, and AWS SageMaker notebooks, running `%pip install` prevents silent environment corruption when multiple Python versions or conda environments coexist on the instance.

For years, the `!pip install` gotcha was one of the single most reported bugs across StackOverflow and GitHub for Jupyter users. In 2019, IPython 7.3 officially introduced the `%pip` and `%conda` magics specifically to eliminate this trap permanently.

• `%pip` reaches the kernel; `!pip` shouts at the shell.
• Always use percent (`%`) for package management inside notebooks.

A Jupyter kernel specification (`kernel.json`) maps an interactive session to a concrete binary:

```json

"argv": [

"/path/to/project/.venv/bin/python",

"-m",

"ipykernel_launcher",

"-f",

"connection_file"

],

"display_name": "Python (llm-scratch)",

"language": "python"

```

Registration procedure:

1. Activate virtual environment: `source .venv/bin/activate`
2. Install ipykernel in environment: `pip install ipykernel`
3. Register the kernel spec:

`python -m ipykernel install --user --name=llm-scratch --display-name "Python (llm-scratch)"`

1. Verify discovery:

`jupyter kernelspec list`

Multi-tenant GPU servers in enterprise research labs host dozens of research projects. Shared JupyterLab servers rely on named kernelspecs so researchers can switch between distinct PyTorch and JAX environments with a single click in the UI.

Fernando Pérez created IPython in 2001, which split in 2014 into the language-agnostic Project Jupyter protocol (ZeroMQ messaging between client and kernel) and the IPython Python kernel (`ipykernel`).

• Register the kernel, name the spec, own your execution.
• A named kernel links Jupyter's frontend to your virtual environment's backend.

The canonical terminal startup protocol:

```bash

# Step 1: Navigate to repository root

cd /path/to/llms-from-scratch

# Step 2: Activate isolated virtual environment

source .venv/bin/activate

# Step 3: Launch JupyterLab directly from active environment

jupyter lab

```

In Notebook Cell 1 (Mandatory Verification Cell):

```python

import sys

import torch

print("Executable:", sys.executable)

assert ".venv" in sys.executable, "ERROR: Not running inside the project virtual environment!"

print("PyTorch Version:", torch.__version__)

print("CUDA Available:", torch.cuda.is_available())

if torch.cuda.is_available():

print("Device Name:", torch.cuda.get_device_name(0))

```

This guarantees that all subsequent cells run against the intended dependencies.

Senior deep learning engineers embed this verification cell as standard boilerplate at the top of every exploratory analysis and model training notebook to prevent silent execution bugs across team members.

The reproducibility crisis in computational science prompted scientific journals (such as Nature and Science) to demand explicit code execution environments. Starting every notebook with deterministic interpreter logging has become an industry gold standard.

• Verify in cell one before you train on step one.
• `sys.executable` never lies.