250 terms from the curriculum, each with what it actually means and the lesson where it shows up.
250 of 250 terms
A training-memory technique that saves only selected forward-pass activations and recomputes the omitted ones during backpropagation.
Careful: Activation checkpointing is not a durable training checkpoint. It helps one forward and backward pass fit in memory but cannot resume a crashed run.
A function applied after a linear or affine layer that introduces nonlinearity. Without it, composing layers with weights and biases collapses to one affine transformation. ReLU, GELU, and SiLU are common choices. The choice directly affects whether gradients flow during training.
Learn it: Activation Functions →Adaptive Moment Estimation. It combines an exponential average of gradients with an exponential average of squared gradients, applies bias correction, and adapts the update scale per parameter. It is a useful baseline, but it still needs a suitable learning rate and schedule.
Careful: Adam is a strong baseline, not a universal best optimizer.
An Adam variant that decouples weight decay from the gradient-based parameter update. That makes the shrinkage behavior easier to reason about than adding an L2 penalty inside Adam's adaptively scaled gradient.
Careful: Decoupled weight decay does not make AdamW universally optimal. Model, data, and training scale still determine the best optimizer and schedule.
A pre-acceptance gate that decides whether a request may enter a bounded queue or service under the system's current capacity, priority, and policy.
Careful: Admission control acts before acceptance. Load shedding can reject or remove work at ingress, in queues, at dependencies, or at other overload boundaries.
A software system that lets a model select actions toward a goal, observe tool or environment results, and continue under an orchestration policy. An agent may use a loop, a state machine, a workflow engine, or human approvals. The model is one component, not the entire system.
Careful: Autonomy is a degree of delegated authority, not a required property of every agent.
Learn it: The Agent Loop: Observe, Think, Act →The runtime around a model that assembles context, exposes tools, manages state, enforces limits, records traces, and decides when the agent should continue, retry, ask, or stop.
Careful: A harness is broader than a prompt template and narrower than the complete product.
Learn it: The Minimal Agent Workbench →Information stored outside the model and selected for use in later agent steps, such as prior decisions, user preferences, task episodes, or verified facts.
Careful: Agent memory is not the same as agent state. State tracks the current run; memory preserves selected information for possible future runs.
A discoverable directory of procedural instructions whose entry point is `SKILL.md`, with optional references, scripts, and assets that a compatible runtime can load in stages.
Careful: Activating a skill supplies context. It does not expose a tool, grant permission, create a sandbox, or prove that the resulting work is correct.
Learn it: Agent Skills: Portable Contract and Runtime Boundary →The explicit data an agent carries across steps, such as the current objective, completed actions, tool results, open questions, budgets, approvals, and artifact references.
Careful: State is not the same as conversation history. A transcript is evidence; state is the compact operational record used to decide what happens next.
Learn it: Repo Memory and Durable State →A documented analysis of how an AI system can affect people, organizations, and environments, including context, hazards, likelihood, impact, controls, residual risk, and monitoring responsibilities.
Careful: A risk assessment supports a decision under stated assumptions. It is not a one-time safety certificate or proof that every hazard has been found.
The effort to make a model or AI system behave in ways that match intended goals, constraints, and human preferences across both expected and adversarial situations.
A control point that blocks a consequential action until an authorized person or policy grants permission.
Careful: An approval gate asks whether an action is authorized. A verification gate asks whether evidence shows the action is correct.
Learn it: Verification Gates →A search method that returns vectors likely to be among the nearest to a query without exhaustively comparing the query with every stored vector.
Careful: ANN describes a search objective and tradeoff, while HNSW is one particular index algorithm that can implement it.
A mechanism that forms contextual representations by comparing query vectors with key vectors, normalizing the resulting scores, and using them to combine value vectors. Masks, position rules, or sparse patterns can restrict which positions participate.
Careful: Attention weights are computation coefficients, not a faithful explanation of model reasoning.
Learn it: Self-Attention from Scratch →A discrete identifier produced by an audio codec or tokenizer for a short segment or feature of an audio signal, sometimes across several codebooks.
Careful: An audio token is not a fixed duration, phoneme, or word. Its meaning and time span depend on the tokenizer and codebook design.
Learn it: Neural Audio Codecs — EnCodec, SNAC, Mimi, DAC and the Semantic-Acoustic Split →A durable, access-controlled record of security- or accountability-relevant events, including who or what acted, what changed, when it happened, and the resulting status.
Careful: A trace helps diagnose one execution path. An audit log preserves events required for accountability across executions and over time.
A system that records or transforms tensor operations so it can compute derivatives, usually with reverse-mode automatic differentiation. You write the forward computation and the framework derives the gradients needed for backpropagation.
Learn it: Chain Rule & Automatic Differentiation →The task and system pipeline that maps a speech signal to a transcription, often with optional token or segment timing and confidence information.
Careful: ASR transcribes what was said. Determining who spoke requires diarization or speaker recognition, while translation and intent understanding are separate tasks.
Learn it: Speech Recognition (ASR) — CTC, RNN-T, Attention →A factorization in which each output token is predicted from the tokens that precede it. During generation, the selected token is appended to the sequence and becomes part of the next prediction's context.
Careful: The unit is a token, not necessarily a word, and generation can use decoding methods other than always selecting the highest-probability token.
A control loop that changes the number or capacity of serving workers from observed demand, resource use, or application metrics within configured bounds.
Careful: Autoscaling adds or removes capacity. It does not make an overloaded dependency faster or guarantee that enough hardware can be acquired in time.
Learn it: GPU Autoscaling on Kubernetes — Karpenter, KAI Scheduler, Gang Scheduling →The proportion of eligible service interactions or time windows in which users can obtain the defined acceptable service under a stated measurement boundary.
Careful: Availability is one reliability outcome. It does not describe latency, correctness, safety, or the experience of every user segment.
A flow-control mechanism that slows or rejects upstream work when a downstream component cannot process it safely at the current rate.
Careful: Backpressure protects capacity before failure. A circuit breaker stops calls after failures show a dependency is unhealthy.
An efficient application of the chain rule that propagates derivatives from a scalar loss backward through a computation graph. It computes gradients; an optimizer uses those gradients to update parameters.
Careful: Backpropagation calculates gradients. It does not choose the update rule or learning rate.
Learn it: Backpropagation from Scratch →The number of examples whose losses contribute to one gradient estimate before an optimizer update. Larger batches can improve hardware utilization and reduce gradient noise, but they require more memory and may need different learning-rate or scheduling choices.
Careful: There is no universal batch-size range or rule that says every batch increase should produce the same learning-rate increase.
Overlap or information leakage between evaluation examples and data used to pretrain, tune, prompt, select, or otherwise improve the evaluated system.
Careful: Contamination is broader than exact copying. Paraphrases, answer keys, benchmark metadata, and repeated prompt tuning can also leak evaluation information.
A lexical ranking function that scores a document from query-term matches while accounting for term rarity, repeated occurrences, and document length.
Careful: BM25 does not understand semantic similarity directly, and its score has no universal meaning across different queries or index configurations.
A subword-tokenization method that repeatedly merges frequent adjacent units to construct a fixed vocabulary from training text.
Careful: BPE is one tokenizer family, not a universal description of how every model creates tokens.
The agreement between a system's stated confidence and the observed frequency with which predictions at that confidence are correct.
Careful: Calibration measures confidence reliability, not overall accuracy, factuality, or reasoning quality.
A deployment strategy that exposes a new version to a limited slice of traffic or infrastructure before expanding the rollout.
Careful: A canary release limits exposure; it does not replace pre-deployment tests, approval, or rollback preparation.
Intermediate reasoning used to decompose a task before producing an answer. A prompt can request a visible rationale, while some systems use internal reasoning that is not returned to the user.
Careful: Chain of thought is not a substitute for tools, tests, or external verification.
Learn it: Few-Shot, Chain-of-Thought, Tree-of-Thought →A durable snapshot used to resume from a known boundary. In a workflow, it stores operational state and artifact references. In model training, it can store parameters, optimizer state, scheduler state, and the training position.
Careful: A workflow checkpoint and a model-training checkpoint serve the same recovery goal but preserve different state. Neither is merely a transcript or a weights file with no resume metadata.
Learn it: Checkpoint Save and Resume →A serving technique that divides a long prompt's prefill work into smaller schedulable pieces so prompt processing can interleave with decode work from other requests.
Careful: Chunked prefill changes how prompt computation is scheduled. It does not split the user's context into independent semantic chunks or change the model's context window.
Learn it: Serving Engine Internals — PagedAttention, Continuous Batching, Chunked Prefill →Dividing source material into retrievable units before indexing. Chunk boundaries, overlap, metadata, and document structure determine whether retrieval returns enough context without flooding the prompt.
A reliability control that temporarily stops calls to a dependency after failures cross a threshold, then probes whether the dependency has recovered.
Careful: A circuit breaker reacts to dependency health. A rate limit controls allowed request volume.
A neural network that uses convolution operations (sliding filters over the input) to detect local patterns. Stacking convolutions detects increasingly complex features: edges, textures, objects.
Careful: Convolutions also work on audio, time series, and other grid-like data.
An agent specialized for software work that can inspect a repository, edit files, run development tools, and use their outputs to advance a scoped engineering task.
Careful: A coding assistant that only suggests text is not necessarily an agent. The agent acts through tools and observes results.
Learn it: Skill Discovery and Progressive Disclosure →A deliberate operation that semantically counteracts a completed side effect when the original operation cannot be rolled back atomically.
Careful: Compensation is a new business action, not time travel. It can fail and therefore needs idempotency, monitoring, and escalation.
Verifiable information about the origin and editing history of a piece of media or other digital content, including the actors, tools, transformations, and assertions attached to it.
Careful: Provenance can establish who asserted a history and whether the record was altered. It does not prove that the depicted event is true or that the content is harmless.
Learn it: Watermarking — SynthID, Stable Signature, C2PA →Reducing the token footprint of source material while attempting to preserve the information required for a later model decision.
Careful: Compression is lossy unless it retains the full original. A shorter summary is not automatically an equivalent context.
Designing the full information environment supplied to a model at each step, including instructions, selected files, retrieved evidence, tool results, examples, state, and output constraints.
Careful: Prompt engineering focuses on instruction wording. Context engineering also decides what evidence and state enter the model's working context.
Learn it: Context Engineering: Windows, Budgets, Memory, and Retrieval →The maximum token capacity available to one model inference under a specific model and API contract. The capacity may include system instructions, messages, retrieved content, tool exchanges, and generated output, with provider-specific accounting and output limits.
Careful: Context is temporary input to an inference. Durable memory is stored outside the model and selected back into later context.
Learn it: Context Engineering: Windows, Budgets, Memory, and Retrieval →A serving scheduler that adds and removes generation requests at iteration boundaries instead of waiting for every request in a fixed batch to finish.
Careful: Continuous batching is an inference scheduling policy, not gradient accumulation or a training batch-size technique.
Training by pulling similar pairs closer and pushing dissimilar pairs apart in embedding space. CLIP uses this: matching image-text pairs vs non-matching ones.
The normalized dot product of two vectors. It compares their direction rather than their magnitude and ranges from -1 to 1 for real-valued vectors.
Careful: High cosine similarity only has meaning relative to the embedding model and the data distribution. It does not prove factual or semantic equivalence.
Total system cost divided by the number of tasks that satisfy a defined success criterion, including retries, failed runs, tool use, and evaluation overhead.
Careful: Cost per token measures usage. Cost per successful task measures useful outcomes.
Attention in which the query representation comes from one sequence or representation while keys and values come from another.
Careful: Cross-attention is not intrinsically multimodal. It can connect two text sequences or other representations; self-attention instead derives queries, keys, and values from the same sequence representation.
A loss based on the negative log probability assigned to the target outcome. In next-token training, it penalizes the model when it assigns low probability to the observed next token.
Careful: Perplexity is the exponentiated average cross-entropy only when the averaging and logarithm base are defined consistently.
NVIDIA's platform and programming model for general-purpose computation on compatible GPUs. Deep-learning frameworks use CUDA libraries and kernels to execute many tensor operations in parallel.
Careful: GPU acceleration is not synonymous with CUDA; other hardware and software stacks exist.
Creating modified examples, such as transformed images, perturbed audio, or paraphrased text, to increase training diversity without collecting entirely new source data. It can reduce overfitting when the transformation preserves the task signal.
Careful: An augmentation must preserve the target label or behavior you want the model to learn.
Assigning data to documented sensitivity or impact classes so handling, access, retention, sharing, and incident rules follow the consequences of disclosure or loss.
Careful: Data classification describes protection requirements. It is not the same as a machine-learning classification task or a claim that the data is accurate.
Detecting and removing exact and near-duplicate examples within or across datasets.
Careful: Deduplication is not ordinary data cleaning. Two distinct records can legitimately share text, and two paraphrases can still carry the same leaked information.
Unauthorized transfer of protected data from a system or trust zone to a person, tool, service, or storage location that is not permitted to receive it.
Careful: Exfiltration is about unauthorized movement or disclosure. Ordinary retrieval of data by an authorized component is not exfiltration, although later use can become one.
Learn it: EchoLeak and the Emergence of CVEs for AI →Unintended use of information during training or feature construction that would not be available at the real prediction point or belongs to a held-out evaluation boundary.
Careful: Leakage is not limited to duplicate rows. Global normalization statistics, timestamps, target-derived features, and repeated test-driven prompt edits can all leak information.
A record of how a data artifact was derived across sources, transformations, joins, filters, versions, and downstream uses.
Careful: Data provenance explains origin and custody broadly. Lineage emphasizes the transformation path and dependencies between data artifacts.
For personal data, limiting what is collected, processed, exposed, and retained to what is necessary for a specified purpose. Teams can apply the same discipline to sensitive non-personal data as an engineering control.
Careful: Minimization does not mean keeping no data. It means being able to justify each data element, use, recipient, and retention period against the stated purpose.
Traceable information about where data originated, who or what transformed it, which versions were used, and how derived artifacts relate to their sources.
Careful: A source URL is only one piece of provenance; it does not describe collection time, licensing, filtering, transformation, or downstream use.
A documented partition of examples into separate subsets for fitting, development decisions, and final evaluation.
Careful: A random split is not automatically independent. Near duplicates, future observations, or records from the same entity can cross the boundary.
Structured documentation of a dataset's motivation, composition, collection process, preprocessing, uses, distribution, maintenance, and known limitations.
Careful: A datasheet documents evidence and intended use. It is not a license, quality guarantee, or substitute for deployment-specific evaluation.
Learn it: Model, System, and Dataset Cards →Passing the remaining end-to-end time budget to downstream calls so each dependency knows how long the original request can still usefully wait.
Careful: A deadline is an absolute or remaining completion boundary. A retry delay controls when another attempt begins and must fit inside that same budget.
The iterative stage of autoregressive inference that generates new tokens one step at a time after the input prefix has been processed.
Careful: Decode phase is not the decoder component of an encoder-decoder model. It names the runtime generation stage.
Learn it: Disaggregated Prefill/Decode — NVIDIA Dynamo and llm-d →A component that maps a representation into an output. In an encoder-decoder transformer, the decoder uses masked self-attention and cross-attention to generate outputs. Decoder-only language models instead generate from a single causal stack.
The algorithm that converts a model's sequence of next-token scores into selected tokens and a completed output.
Careful: Decoding changes how outputs are selected; it does not change the model's trained parameters or add knowledge.
Using independent preventive, detective, and corrective controls at several system boundaries so one failed control does not determine the outcome.
Careful: More controls are not automatically better. Layers should address distinct failure modes and remain testable rather than repeat the same assumption.
Assigning a bounded subtask to another person or agent together with the needed context, authority, output contract, and return conditions.
Careful: Sending a vague message to another agent is not reliable delegation. The receiver needs a scope contract and a defined handoff back.
First-stage retrieval that embeds queries and candidates into vector representations and ranks candidates by a similarity function.
Careful: Dense retrieval is not a reranker. It searches the collection, while a reranker rescores a smaller candidate set.
A generative model trained around a progressive noising process and a learned reverse process. Sampling usually begins from noise and applies repeated denoising steps, sometimes in a learned latent space.
Careful: Diffusion is a general generative framework, not an image-only technique.
A serving architecture that runs prefill and decode work in separately provisioned worker pools and transfers the required attention state between them.
Careful: Disaggregation separates runtime stages. It does not split one model into tensor or pipeline-parallel shards within a stage.
Learn it: Disaggregated Prefill/Decode — NVIDIA Dynamo and llm-d →A difference between the data distribution used to build or evaluate a system and the distribution it encounters after deployment.
Careful: Distribution shift is not always model drift. The model may be unchanged while its environment or user population changes.
A preference-optimization objective that trains a policy directly from preferred and rejected response pairs relative to a reference policy. It avoids running an explicit reward model and reinforcement-learning loop during this stage.
Careful: DPO still depends on the quality and coverage of preference data and does not eliminate evaluation or alignment risk.
Learn it: DPO: Direct Preference Optimization →During training, randomly setting a fraction of activations to zero encourages the network not to rely on one activation path. It is normally disabled for standard inference, although Monte Carlo dropout deliberately keeps it active to estimate uncertainty.
Running a workflow so its state and completed steps survive process crashes, restarts, or long waits without redoing confirmed side effects.
Careful: Durable execution does not make every operation safe automatically. Side effects still need idempotency and compensation rules.
A runtime policy that forms inference batches from queued requests according to compatible shapes, maximum size, priority, and allowed queue delay.
Careful: Dynamic batching assembles batches from queued work. Continuous batching changes membership while autoregressive generation is already running.
Learn it: Serving Engine Internals — PagedAttention, Continuous Batching, Chunked Prefill →Combining raw or low-level representations from several modalities before most task-specific modeling occurs.
Careful: Early fusion describes where streams are combined in the architecture. It does not guarantee that the model learns useful alignment between them.
Learn it: Chameleon and Early-Fusion Token-Only Multimodal Models →A scalar that describes how a linear transformation scales a corresponding nonzero eigenvector without changing its direction. In covariance-matrix PCA, larger eigenvalues correspond to directions with more variance.
A learned mapping from discrete items (words, images, users) to dense vectors in continuous space, where similar items end up close together
Careful: Similarity depends on the model, training objective, and metric. Distance in one embedding space does not carry over to another.
Learn it: Embeddings & Vector Representations →A component that transforms input into a representation. A transformer encoder commonly uses non-causal self-attention, subject to any masks, so each position can incorporate context from across the input.
Careful: Encoder-only models can produce outputs through task heads even though they are not typically used for autoregressive text generation.
One traversal of the defined training dataset. In distributed or sampled training, the exact implementation of an epoch depends on the data loader and sampling policy.
Careful: More epochs do not guarantee better generalization; evaluate on held-out data.
The amount of unsuccessful service allowed by a service-level objective over its measurement window before the objective is exhausted.
Careful: An error budget is not a quota for causing incidents. It is an operating policy derived from a user-facing reliability target.
A versioned collection of inputs, expected properties, scoring rules, and metadata used to measure an AI system against a defined capability or risk.
Careful: A development eval guides iteration, a final held-out test estimates performance after choices are fixed, and a standardized benchmark supports comparison under a shared protocol. Repeated tuning against any held-out set leaks test information and inflates results.
Learn it: Eval-Driven Agent Development →A defined process for measuring model or system behavior on representative tasks using explicit success criteria, data, scorers, and review procedures.
Careful: A benchmark score is one evaluation result, not a complete account of production quality.
Learn it: Evaluation & Testing LLM Applications →A metric that counts an output as correct only when its normalized representation exactly equals an accepted reference answer.
Careful: A low exact-match score can reflect harmless formatting differences, while a matching string can still be unsupported or unsafe in context.
Distributing mixture-of-experts subnetworks across devices and routing each token's activations to the devices that host its selected experts.
Careful: Expert parallelism partitions experts selected by a router. Tensor parallelism partitions the tensor operations inside layers.
Learn it: Mixture of Experts (MoE) →An individual measurable property of the data. In classical ML, you engineer features by hand. In deep learning, the network learns features automatically from raw data.
Careful: A stored column can contain several useful features, and a learned representation can contain features with no simple human label.
In-context learning that includes a small set of demonstrations before the target input so the model can infer the desired task, format, or decision boundary.
Continuing training from pretrained parameters on a narrower dataset or objective. Depending on the method, you may update all parameters, selected parameters, or added adapter parameters.
Careful: Fine-tuning can influence encoded knowledge, but it does not simply append records to a searchable database inside the model.
Learn it: Fine-Tuning with LoRA & QLoRA →A test that can pass and fail across equivalent runs without a relevant change to the code or intended test environment.
Careful: A test that consistently exposes an intermittent product bug is valuable evidence, not necessarily a flaky test.
An exact attention algorithm that tiles the computation to reduce transfers between accelerator memory levels while avoiding materialization of the full attention matrix in high-bandwidth memory.
Careful: FlashAttention changes how attention is computed, not the mathematical attention result it targets. It is separate from KV caching and quantization.
Learn it: KV Cache, Flash Attention & Inference Optimization →A provider or application interface through which a model emits a structured request naming a tool and its arguments. Application code validates the request, performs the operation, and can return the result for another model step.
Careful: The model requests a function call; your trusted code decides whether and how to execute it. Function calling alone is not a complete agent.
Learn it: Function Calling & Tool Use →A generator network tries to create realistic data while a discriminator network tries to tell real from fake. They train together: the generator gets better at fooling the discriminator, and the discriminator gets better at detecting fakes.
The rate of completed requests that satisfy defined service constraints, such as both time-to-first-token and per-token latency objectives, under a stated workload.
Careful: Goodput is not all completed throughput and is not a universal property of a model. It depends on workload and success thresholds.
Learn it: Inference Metrics — TTFT, TPOT, ITL, Goodput, P99 →Generative Pre-trained Transformer, a family label for generative transformer models pretrained on sequence-prediction objectives and adapted for downstream use. Product names and model architectures should not be treated as interchangeable.
Preserving a bounded core service when capacity or dependencies are impaired by reducing optional quality, features, freshness, or workload instead of failing every request.
Careful: Graceful degradation is not silently returning a worse answer as if nothing happened. Operators always need visibility; users need disclosure when the reduced mode materially changes the result or service contract.
Learn it: Building a Production LLM Application →A vector of partial derivatives pointing in the direction of steepest increase. In ML, you go opposite to the gradient (gradient descent) to minimize the loss.
Careful: Optimizers can transform, average, clip, or adapt gradients instead of taking a plain negative-gradient step.
Summing or averaging gradients from several microbatches before performing one optimizer update.
Careful: Gradient accumulation reduces per-step activation memory, but it does not reproduce every property of processing the full batch simultaneously.
Limiting gradient values or their combined norm before an optimizer update when they exceed a chosen threshold.
Careful: Clipping controls update magnitude; it does not repair invalid data, a broken loss, or a consistently unsuitable learning rate.
A family of optimization updates that move parameters using the negative gradient of an objective, usually estimated from batches rather than the entire dataset.
Connecting a generated answer or action to evidence, state, or observations that the system can identify and check.
Careful: Adding documents to a prompt creates an opportunity for grounding. It does not guarantee the model will use them correctly.
Learn it: RAG (Retrieval-Augmented Generation) →System controls that constrain inputs, tool use, outputs, permissions, and escalation. They can include schemas, policy checks, classifiers, allowlists, sandboxing, approvals, and post-action verification.
Careful: Guardrails reduce risk; they do not prove that an AI system is safe.
Learn it: Guardrails, Safety & Content Filtering →Generated content that is false, unsupported by the available evidence, or inconsistent with the task's source of truth. It can arise even when the output is fluent and the model is not attempting to deceive.
Careful: A hallucination is an output-quality failure, not a diagnosis of model intent.
A structured transfer of a task between people or agents that preserves the objective, current state, evidence, decisions, constraints, and remaining work.
Careful: A summary says what happened. A handoff also says what state is authoritative and what should happen next.
Learn it: Multi-Session Handoff →An approximate-nearest-neighbor index that organizes vectors in layered proximity graphs and searches from coarse upper layers toward detailed lower layers.
Careful: HNSW is an index algorithm, not a similarity metric, embedding model, or complete vector database.
A workflow design in which a person supplies judgment, correction, approval, or escalation at defined points in an AI-driven process.
Careful: HITL does not automatically make a system safe. Reviewers need time, context, authority, and a clear decision standard.
Retrieval that combines signals from different methods, commonly lexical matching and dense-vector similarity, before merging or reranking results.
Careful: Hybrid retrieval combines candidate signals. A reranker applies a second relevance model to candidates already retrieved.
Learn it: Advanced RAG (Chunking, Reranking, Hybrid Search) →A configuration choice that shapes model structure, optimization, data processing, or inference rather than being learned as an ordinary model parameter. Examples include learning rate, batch size, layer count, and decoding settings.
Careful: Some hyperparameters are selected before training, while others can be changed during a schedule or at inference time.
The property that repeating the same operation with the same identity does not create additional side effects beyond the first successful application.
Careful: Idempotency does not mean every response is byte-for-byte identical. It means the intended state change is not duplicated.
A model-specific visual unit represented as a vector or discrete code, commonly derived from an image patch, region, or learned visual-codebook entry.
Careful: An image token is not necessarily one pixel, one object, or one fixed physical area. Its scope follows the visual encoder or tokenizer.
Learn it: Vision-Language Models — The ViT-MLP-LLM Pattern →A model adapting its behavior from instructions, examples, or patterns supplied in the current input without an ordinary parameter update.
Careful: In-context learning is temporary conditioning, not fine-tuning, durable memory, or proof that the model inferred the intended rule.
The coordinated process for detecting, analyzing, containing, recovering from, communicating, and learning from an event that threatens service, data, safety, or security.
Careful: Incident response manages the event and its consequences. Root-cause analysis and long-term prevention continue after immediate service is restored.
Learn it: SRE for AI — Multi-Agent Incident Response, Runbooks, Predictive Detection →A prompt-injection attack delivered through content the system retrieves or observes, such as a webpage, document, email, image text, or tool result, rather than directly through the user's instruction.
Careful: Indirect describes the delivery path, not a weaker attack. A hidden instruction in retrieved content can be as consequential as a direct user prompt.
Learn it: Indirect Prompt Injection — Production Attack Surface →Structural or statistical assumptions that favor some functions or representations over others. Convolution favors locality and shared filters; causal masking favors prediction from preceding positions.
Careful: Transformers still have inductive biases through tokenization, position handling, masking, architecture, data, and objective.
Executing a trained model to produce predictions, scores, embeddings, or generated tokens without performing an ordinary training update to its parameters.
Careful: An application can update caches, conversation state, or external memory during inference even though model weights stay unchanged.
A model capability to map natural-language directions and supplied context to behavior that satisfies the stated task and constraints.
Careful: Instruction following is not factual correctness, alignment, or obedience to every string that looks like an instruction.
A rule set for resolving conflicts among instructions from sources with different authority, such as application policy, users, and untrusted retrieved content.
Careful: An instruction hierarchy can improve behavior but is not a security boundary; least privilege and approval controls still limit consequences.
The elapsed time between two consecutive output-token arrival events for one request, calculated as `t_i - t_(i-1)` for an output token after the first.
Careful: ITL is one interval between consecutive tokens. Time per output token is a per-request average across those intervals, while time to first token covers the wait before streaming begins.
Learn it: Inference Metrics — TTFT, TPOT, ITL, Goodput, P99 →An adversarial input or interaction strategy intended to make a model produce behavior that its training or application controls are designed to prevent.
Careful: A jailbreak targets model or system behavioral restrictions. Prompt injection redirects instruction following, often toward an attacker's goal; one interaction can involve both.
Learn it: Capstone 82 — Jailbreak Taxonomy →A Python library for transforming numerical functions with automatic differentiation, compilation, vectorization, and parallel execution across accelerators. Its transformations work best with explicit state and functional-style code.
Careful: JAX does not prohibit all stateful programming, but hidden mutation inside transformed functions can produce incorrect or unsupported behavior.
Learn it: Introduction to JAX →Training a student model to reproduce selected behavior or output distributions from a more capable teacher, often alongside ordinary target labels.
Careful: Distillation transfers behavior on the training distribution; it does not copy every capability, fact, or safety property of the teacher.
Stored key and value tensors from earlier positions in autoregressive generation. Reusing them avoids recomputing attention projections for the unchanged prefix at every decoding step.
Careful: A KV cache is runtime attention state for a sequence. Prefix caching reuses eligible KV state across requests, while prompt caching is a broader provider or application reuse contract.
Learn it: KV Cache, Flash Attention & Inference Optimization →Processing modalities through separate encoders or predictors and combining their high-level representations, scores, or decisions near the task output.
Careful: Late fusion describes the position of combination. It does not mean simple averaging or guarantee that the modalities contribute equally.
Learn it: Cross-Attention Fusion →A learned representation space whose coordinates encode factors useful to a model. It may be lower-dimensional than the input, but compression is not required for every latent representation.
Careful: Nearby points are only meaningfully similar according to what the model and training objective learned.
A scale factor used by an optimizer to control parameter-update magnitude. Values that are too large can destabilize training; values that are too small can make useful progress impractically slow.
Careful: The effective update also depends on the optimizer, schedule, gradient scale, batch, and parameter history.
A policy that changes the optimizer's learning rate as training progresses according to steps, epochs, metrics, or a predefined curve.
Careful: A scheduler controls the learning rate over time; it does not decide when an optimizer step occurs or guarantee convergence.
Giving a model, agent, tool, or user only the permissions required for the current task, for only as long as those permissions are needed.
Careful: Authentication proves identity. Least privilege limits what that identity can do.
A language model with enough capacity and broad training to perform many language tasks through prompting or adaptation. Most current LLMs use transformer architectures and sequence-prediction objectives, but size thresholds, data sources, and training recipes vary.
Careful: An LLM is a model component. Tools, retrieval, state, policies, and product logic live in the surrounding system.
Using a language model to score, compare, classify, or critique another system's output against a rubric.
Careful: A judge model is not ground truth. It can be biased by order, verbosity, style, prompt wording, or shared model failures.
Learn it: Eval-Driven Agent Development →Deliberately rejecting, dropping, or cancelling selected work at one or more overload boundaries when demand exceeds the capacity available to produce useful results.
Careful: Load shedding is not confined to work that has already been accepted. Admission control is specifically the pre-acceptance gate, while rate limiting can enforce a usage policy even when capacity remains.
The model's unnormalized numeric scores for candidate outcomes before a normalization function or decoding rule converts them into selections.
Careful: Logits are not probabilities and are not comparable across unrelated positions, models, or tasks without a defined transformation.
A method that keeps base weights frozen and learns low-rank update matrices for selected layers. It reduces the number of trainable parameters and can lower training memory relative to full-parameter fine-tuning.
Careful: Actual memory and speed savings depend on rank, target modules, optimizer state, activation memory, quantization, and implementation.
Learn it: Fine-Tuning with LoRA & QLoRA →An objective that maps predictions and targets, sometimes with regularization terms, to a value optimization tries to reduce. The loss determines which errors training directly rewards or penalizes.
Careful: A low training loss does not guarantee useful, safe, or generalizable behavior on production tasks.
A long-context failure pattern in which model performance changes with evidence position and can degrade when relevant information sits between the beginning and end.
Careful: It is an observed behavior pattern, not a fixed law that affects every model, task, or position identically.
A selection rule that balances relevance to the query with novelty relative to items already selected.
Careful: MMR diversifies an existing candidate set; it does not retrieve missing evidence or prove that selected passages are correct.
An open JSON-RPC protocol for a host to connect to servers that expose tools, resources, prompts, and extensions through defined request, result, discovery, and transport contracts. In revision 2026-07-28, every request carries its protocol version and client capabilities instead of relying on an initialization handshake or protocol session.
Careful: MCP standardizes discovery and exchange. It does not decide which tool is safe to call, grant permission, or forbid an application from using explicit state handles.
Learn it: Model Context Protocol (MCP) →An attack that estimates whether a particular record or example was included in a model's training data by observing model outputs or other accessible signals.
Careful: Membership inference asks whether a record participated in training. Model extraction tries to reproduce model behavior, while direct memorization tests whether content can be recovered.
Learn it: Differential Privacy for LLMs →A numerical strategy that uses different data types for different operations, often lower precision for many matrix operations and higher precision for values that need more range or stability.
Careful: Speed, memory, and accuracy effects depend on hardware, data type, scaling method, kernels, and model. They are not a fixed multiplier.
A form of information with its own structure and acquisition process, such as text, image, audio, video, depth, or sensor measurements.
Careful: A modality is not merely a file extension or feature column. Several encodings can represent one modality, and one sample can contain several modalities.
Learn it: MIO and Any-to-Any Streaming Multimodal Models →Learning or establishing correspondences between representations from different modalities so semantically or temporally related items can be matched.
Careful: Alignment makes representations comparable or corresponding. It does not require them to become identical or erase modality-specific information.
Learn it: Projection Layer for Modality Alignment →A structured report describing a model's intended uses, evaluation conditions, performance characteristics, limitations, and relevant ethical or safety considerations.
Careful: A model card communicates evidence and limitations; it is not a certification, warranty, system threat model, or substitute for deployment-specific evaluation.
A component that selects a model or provider for a request using requirements such as capability, latency, cost, context size, policy, and current availability.
Careful: Routing is a policy decision. Random load balancing only distributes traffic.
The runtime and API layer that loads versioned model artifacts, accepts inference requests, schedules execution, manages resources, and returns results under an operational contract.
Careful: Model serving is broader than calling inference once and narrower than the complete application, which may also include retrieval, tools, policy, and user state.
Learn it: Self-Hosted Serving Selection — Matching Engine to Hardware and Scale →An architecture with multiple expert subnetworks and a learned router that selects a subset for each input unit, often each token. Sparse activation can increase total parameter capacity without using every expert on every forward pass.
Careful: Product names do not prove an MoE architecture unless the model developer discloses it.
Learn it: Mixture of Experts (MoE) →An MCP request pattern in which an operation returns `resultType: input_required` with one or more `inputRequests`, then the client retries the original method with `inputResponses` and the exact returned `requestState`.
Careful: `requestState` is untrusted round-trip data. Integrity-protect it before using it for authorization or business decisions, and do not treat it as a server-side session identifier.
Learn it: Explicit Scope and Stateless Elicitation →Combining evidence or learned representations from more than one modality to produce a joint representation, prediction, or generated output.
Careful: Fusion is the combination operation. Alignment establishes correspondence, and merely placing two modalities in one request does not prove either occurred successfully.
Learn it: Cross-Attention Fusion →A model that learns from, relates, or generates more than one modality through representation, alignment, fusion, translation, or coordinated prediction.
Careful: A pipeline with separate image and text models is multimodal at the system level, but it is not necessarily one jointly trained multimodal model.
Learn it: MIO and Any-to-Any Streaming Multimodal Models →A floating-point value representing an undefined or unrepresentable numerical result. In training, NaNs can come from invalid operations, overflow, unstable normalization, excessive updates, or earlier corrupted values.
A family of transformations that rescale or recenter inputs, activations, or features using defined statistics. Batch normalization and layer normalization use different axes and behave differently across training and inference.
Careful: Normalization can improve optimization stability, but it does not always permit a larger learning rate or improve every architecture.
A decoding method that samples from the smallest set of next-token candidates whose cumulative probability reaches a chosen threshold.
Careful: Top-p is a probability-mass threshold, while top-k always keeps a fixed maximum number of candidates.
The ability to understand an AI system's behavior from recorded inputs, outputs, state transitions, tool calls, timings, costs, errors, and evaluation signals.
Careful: Logging collects events. Observability makes those events structured and connected enough to answer operational questions.
Learn it: Agent Observability: Langfuse, Phoenix, Opik →An algorithm that transforms gradients into parameter updates. Plain stochastic gradient descent is a simple baseline; momentum, Adam, and other optimizers change the update using history or adaptive scaling. Each choice has different memory, stability, and tuning behavior.
Careful: The optimizer consumes gradients; backpropagation computes them.
The control logic that sequences, branches, delegates, retries, pauses, resumes, and terminates work across model and tool steps.
Careful: Orchestration is not synonymous with autonomy or multi-agent systems; one agent can be orchestrated through a deterministic workflow.
A generalization gap in which performance on training data is substantially better than performance on representative unseen data. Memorization can contribute, but the operational symptom is poor generalization.
A KV-cache memory manager that stores attention state in fixed-size blocks and maps logical sequence positions to physical blocks instead of requiring one contiguous allocation per sequence.
Careful: Paged KV cache manages runtime attention-state memory. It does not move model parameters to disk or extend the model's trained context limit.
Learn it: Serving Engine Internals — PagedAttention, Continuous Batching, Chunked Prefill →A value learned during training, commonly a weight, bias, embedding element, or normalization parameter. Parameter count is one measure of model capacity, but it does not directly determine quality, memory, or serving cost.
Careful: Memory per parameter depends on numerical format, quantization metadata, sharding, optimizer state, activations, and runtime overhead.
Across a task set, the fraction of tasks for which at least one of k sampled candidates passes a defined correctness test.
Careful: Pass@k is not single-attempt accuracy, and a higher score can reflect a larger attempt budget rather than a better first answer.
A reviewable representation of changes to one or more files, usually expressed as additions and deletions against a known base revision.
Careful: A patch captures file changes, not the reasoning, test evidence, or approval needed to ship them.
Learn it: The Workbench on a Real Repo →A learned projection that converts an image patch into a fixed-width vector used as one element of a transformer input sequence.
Careful: A patch embedding is the vector representation of a patch, not a semantic object detector or a guarantee that patch boundaries match visual entities.
Learn it: Vision Transformers and the Patch-Token Primitive →The exponentiated average negative log-likelihood under a stated tokenization and logarithm convention. Lower values mean the model assigned higher probability to the evaluated sequence.
Careful: Perplexity is not comparable across different tokenizers or evaluation setups and does not directly measure factuality or usefulness.
Partitioning sequential groups of model layers across devices and moving microbatches or requests through those stages as a pipeline.
Careful: Pipeline parallelism divides layers by depth. Tensor parallelism divides tensor operations within a layer.
Learn it: Scaling: Distributed Training, FSDP, DeepSpeed →Constructing, selecting, or revising a sequence of actions and dependencies intended to move from the current state to a goal.
Careful: A generated plan is a proposal, not proof that the steps are feasible, sufficient, or safe.
A durable incident record that explains impact, detection, response, contributing conditions, recovery, and owned follow-up actions without assigning blame as a substitute for analysis.
Careful: A postmortem is not a meeting transcript or a search for one person's mistake. It should produce testable system improvements.
Precision asks how many flagged items were correct; recall asks how many relevant items were found. When you change the decision threshold for one fixed scoring model, improving recall often lowers precision and vice versa. A better model can improve both. F1 is their harmonic mean.
Careful: The right threshold and metric depend on the cost of each error and the prevalence of the target class.
The initial inference stage that processes all supplied input tokens to produce their representations and the attention state required for subsequent autoregressive generation.
Careful: Prefill is the runtime prompt-processing stage, not the first generated token itself. The first token appears only after prefill and any queueing complete.
Learn it: Disaggregated Prefill/Decode — NVIDIA Dynamo and llm-d →Reusing KV-cache blocks produced for an identical eligible token prefix across requests so the serving runtime can skip repeated prefix computation.
Careful: Prefix caching reuses runtime attention state for exact token prefixes. Prompt caching is a broader provider or application contract, while semantic caching reuses a prior result for a similar request.
Learn it: Inference Optimization →Supplying a person or model with the minimum useful context first, then revealing deeper detail when the task or evidence requires it.
Careful: Progressive disclosure is staged access to detail, not deliberate withholding of information required for a decision.
Learn it: The Workbench on a Real Repo →Reuse of provider-side or application-side computation for an identical or eligible prompt prefix so repeated inference avoids some preprocessing work.
Careful: A prompt cache is a provider or application reuse contract and may use prefix caching internally. Prefix caching specifically reuses eligible exact-token KV state, while semantic caching reuses a prior result for a sufficiently similar request.
Learn it: Prompt Caching and Context Caching →Designing model-facing instructions, examples, constraints, and output requirements to improve behavior on a defined task.
Careful: Prompt wording cannot compensate for missing evidence, unsafe permissions, poor tool contracts, or absent evaluation.
Learn it: Prompt Engineering: Techniques & Patterns →An attack or failure mode in which untrusted content influences a model to disregard intended instructions, expose data, misuse tools, or take actions outside the user's goal. The content can arrive directly from a user or indirectly through retrieved pages, files, messages, or tool output.
Careful: Prompt injection is not technically the same mechanism as SQL injection, and a stronger system prompt is not a complete defense.
Learn it: Prompt Injection and the PVE Defense →Variation in model output or measured performance caused by changes to prompt wording, order, formatting, or examples that preserve the intended task.
Careful: Sensitivity is not always a prompt defect; it can reveal ambiguity, weak model robustness, unstable decoding, or an inadequate scoring rule.
Authenticated, machine-readable metadata that binds an artifact to claims about how, where, when, and from which inputs it was produced.
Careful: A signature identifies the attester and protects integrity; it does not prove that every claim inside the attestation is true.
For personal data, collecting and using it only for specified, explicit purposes unless a new use has an appropriate compatible or authorized basis.
Careful: Purpose limitation governs why data is used. Data minimization governs how much data that purpose actually requires.
A parameter-efficient fine-tuning method that keeps a pretrained base model frozen in a low-bit quantized representation while training LoRA adapters with higher-precision computation where needed.
Careful: QLoRA does not guarantee a particular memory footprint or a fixed quality gap from full fine-tuning.
Learn it: Fine-Tuning with LoRA & QLoRA →Representing weights, activations, or caches with lower-precision formats to reduce memory, bandwidth, or compute cost. Methods differ in calibration, granularity, data type, and whether conversion happens before, during, or after training.
Careful: Moving from one nominal bit width to another does not guarantee the same end-to-end memory or speed ratio because metadata, kernels, caches, and hardware support also matter.
A system pattern that retrieves evidence relevant to a request and supplies selected content to a generative model before it answers or acts. Retrieval can use lexical, vector, structured, or hybrid methods.
Learn it: RAG (Retrieval-Augmented Generation) →A policy that caps requests, tokens, concurrent work, or another resource within a defined time or capacity window.
Careful: A rate limit controls allowed usage. Backpressure propagates downstream capacity constraints through a system.
An agent pattern that interleaves task reasoning, a concrete action, and an observation returned by the environment before deciding the next step.
Careful: ReAct is a prompting and control pattern, not a guarantee of autonomy, correctness, or safe tool use.
A diagnostic that tells the traffic-routing layer whether a service instance is currently able to accept requests.
Careful: Readiness controls traffic eligibility. Liveness decides whether the process should be restarted, and neither proves that every model response will be correct.
Learn it: Building a Production LLM Application →For one query, Recall@K is `|relevant items intersecting the top k| / |relevant items|`. A dataset score aggregates those per-query values under a stated rule.
Careful: High Recall@K does not mean the top result is good, the ranking is well ordered, or the final answer is grounded. Queries with no relevant items require an explicit exclusion or assigned-value policy because the denominator is zero.
A rank-fusion method that combines several result lists by summing contributions that decrease with each item's rank in each list.
Careful: RRF combines ranks, not embeddings or relevance scores, and it cannot recover an item absent from every input list.
A structured adversarial testing process in which authorized testers seek failures using documented objectives, threat assumptions, cases, and evidence.
Careful: A list of jailbreak prompts is not a complete red-team program, and red teaming cannot prove the absence of unknown failures.
A repeatable check that protects behavior known to work, especially after code, prompt, model, retrieval, or tool changes.
Careful: A regression test guards a specific expected behavior. A broad benchmark estimates performance across a wider task distribution.
Learn it: Eval-Driven Agent Development →Rectified Linear Unit, defined as `f(x) = max(0, x)`. It is inexpensive and has a non-saturating positive branch, though zero gradients on negative inputs can create inactive units.
Version-controlled guidance that tells coding agents how a repository is organized, which commands and conventions apply, what boundaries to respect, and how to verify work.
Careful: Repository instructions complement source code and human documentation; they do not override the user's current request or guarantee that an agent follows them correctly.
A compact, maintained description of a repository's important directories, ownership boundaries, entry points, build commands, tests, generated files, and local instructions.
Careful: A raw file tree shows names. A repository map explains which paths matter and how they relate to a task.
Learn it: Repo Memory and Durable State →A build whose declared source, environment, and instructions can be independently rerun to produce bit-for-bit identical specified artifacts.
Careful: A build that succeeds twice is repeatable evidence, but reproducibility requires the declared independent conditions and identical outputs.
A second-stage model or scoring function that reorders a small candidate set using a richer comparison between the query and each candidate.
Careful: A reranker does not search the entire corpus. It only reorders candidates that retrieval already found.
A bound on retry traffic, usually expressed relative to original requests or over a time window, that prevents retries from consuming unbounded capacity.
Careful: A retry budget limits extra attempts. An error budget measures user-visible unreliability allowed by an SLO.
Repeating a failed transient operation after progressively longer delays, usually with randomized jitter and a strict retry limit.
Careful: Do not retry permanent validation or permission errors, and do not retry non-idempotent operations without a duplication strategy.
An agent assigned to inspect another agent's artifact or decision against explicit criteria and return findings or a verdict.
Careful: A second model call is not automatically independent or correct. Shared context, model bias, and vague criteria can reproduce the same mistake.
Learn it: Reviewer Agent: Separate Builder from Marker →A family of pipelines that uses human feedback to learn a reward or preference signal and then optimizes a model policy against that signal. Implementations vary and need not all use the same reinforcement-learning algorithm.
Careful: RLHF optimizes a proxy learned from collected feedback. It does not guarantee broad alignment with every user or situation.
Learn it: RLHF: Reward Model + PPO →Restoring a previously known deployment or configuration when the current release violates operational, quality, or safety criteria.
Careful: Code rollback does not automatically reverse database migrations, external side effects, cached outputs, or data written by the bad release.
A family of metrics that compares generated text with reference text using units such as n-gram overlap or longest common subsequence.
Careful: Surface overlap can miss semantic equivalence and can reward copied wording without proving factual quality.
An isolated execution environment that restricts an agent's access to files, processes, network destinations, credentials, and host resources.
Careful: A sandbox reduces impact. It does not establish that the code inside is correct or harmless.
Learn it: Production Runtimes: Queue, Event, Cron →The degree to which a constrained resource or service has exhausted its capacity, including queued work that cannot begin promptly.
Careful: Saturation is not one universal percentage. The limiting resource and its queueing behavior depend on the workload and architecture.
A concrete agreement that defines a task's goal, allowed and forbidden surfaces, expected artifacts, verification requirements, and stopping conditions.
Careful: A task description says what you want. A scope contract also defines boundaries and proof.
Learn it: Scope Contracts and Task Boundaries →Attention in which queries, keys, and values are derived from the same sequence representation. Scaled similarity scores are normalized and used to combine values, subject to causal, padding, local, or other masks.
Careful: Not every token can always attend to every other token. Causal and sparse models intentionally restrict connections.
Learn it: Self-Attention from Scratch →A cache that reuses a previous result when a new request is judged sufficiently similar under a chosen representation and threshold.
Careful: Semantic similarity does not guarantee that two requests have the same correct answer. A semantic cache reuses a prior result, while prefix caching reuses exact-token KV state and prompt caching follows provider or application eligibility rules.
Retrieval that represents a query and candidates in an embedding space and ranks candidates using a vector-similarity function.
Dividing conflicting responsibilities or authority across independent roles so one principal cannot complete a high-risk action without another authorized decision.
Careful: Separation of duties is about conflicting authority, not simply assigning work to several people or agents that share the same credentials.
A quantitative measure of service behavior at a defined user-relevant boundary, such as successful request ratio or latency below a threshold.
Careful: An SLI is the measurement. An SLO is the target applied to that measurement over a defined period.
A target range or threshold for a service-level indicator over a stated population and measurement window.
Careful: An SLO is an internal reliability objective. A contractual service-level agreement can include remedies and may use different definitions.
Learn it: Inference Metrics — TTFT, TPOT, ITL, Goodput, P99 →Fine-tuning a pretrained model on paired inputs and desired responses so it learns the demonstrated behavior under the training distribution.
Careful: SFT can adapt many behaviors beyond chat, and example quality determines what behavior is reinforced.
A copy of live request traffic sent to a candidate system for observation while the candidate response remains outside the primary user response path. Because the copied request still executes, its side effects must be isolated.
Careful: Keeping a candidate response off the primary path does not make execution side-effect-free. A canary release differs because it serves real users from the candidate for a controlled share of traffic.
Learn it: Shadow Traffic, Canary Rollout, and Progressive Deployment for LLMs →A common vector space in which representations from different modalities can be compared with the same similarity function.
Careful: Sharing a vector dimension does not create a shared semantic space. The training objective and data must establish cross-modal comparability.
Learn it: CLIP and Contrastive Vision-Language Pretraining →The complete installable skill directory, including `SKILL.md` and every reference, script, asset, fixture, or companion file required by the workflow.
Careful: `SKILL.md` is the entry point, not necessarily the entire artifact.
Learn it: Skill Evals, Packaging, and Portability →The compact model-visible inventory of eligible skills, usually containing routing metadata such as name, description, and an internal source identifier rather than every skill body.
Careful: A catalog entry means the skill is discoverable. It does not mean the body is active or its tools are authorized.
Learn it: Skill Discovery and Progressive Disclosure →A runtime pipeline that searches configured roots, identifies candidate skill directories, validates their package contract, attaches scope and provenance, resolves collisions, and publishes eligible catalog entries.
Careful: Skill discovery is not an unrestricted recursive search for filenames called `SKILL.md`; installation locations and precedence are runtime policy.
Learn it: Skill Discovery and Progressive Disclosure →The runtime-mediated process in which an eligible human, model, application, or other skill selects a skill and causes its instructions to enter the working context.
Careful: Invocation activates instructions. It does not automatically execute a command or bypass approval and sandbox policy.
Learn it: Skill Invocation and Routing →A function defined by `softmax(x_i) = exp(x_i) / sum(exp(x_j))`, implemented with numerical stabilization. Its outputs are positive and sum to one, so they can parameterize a categorical distribution.
Careful: Softmax values are not automatically calibrated probabilities about real-world correctness.
A structured inventory of software components and relationships associated with a product or artifact, often including versions, suppliers, licenses, and identifiers.
Careful: An SBOM is an inventory, not proof that components are secure, correctly licensed, or actually present unless generation and provenance are trustworthy.
An inference method in which a cheaper draft process proposes several tokens and the target model scores those draft positions in parallel. In exact sampling variants, an acceptance and correction rule preserves the target model's output distribution.
Careful: Speculative decoding is not ordinary model routing or unverified autocomplete. Exact variants preserve the target distribution through acceptance and correction, while approximate variants may trade that guarantee for speed.
The MCP 2026-07-28 request model in which every request carries the protocol version and client capabilities in `params._meta`, while results carry an explicit `resultType`; no protocol state is keyed by an initialization handshake, connection, or `Mcp-Session-Id`.
Careful: Stateless MCP removes protocol sessions, not application state, transport connections, streaming responses, tasks, or explicit handles.
Learn it: MCP Fundamentals: Stateless Requests and JSON-RPC →An optimizer family that updates parameters from a gradient estimated on a sampled example or minibatch rather than the complete training dataset.
Careful: In current practice, SGD usually means minibatch SGD, and its useful learning rate does not follow one universal batch-scaling rule.
An application-specified token or text pattern that causes generation to stop when the decoding system encounters it.
Careful: A stop sequence is a mechanical decoding condition, not proof that the answer is complete or that an agent goal is satisfied.
Delivering incremental response events before the complete result is ready. A stream may contain token text, structured deltas, tool-call arguments, usage metadata, or status events depending on the API.
Careful: Network transport, event shape, and chunk boundaries are provider-specific and are not guaranteed to align with words or tokens.
Learn it: Building a Production LLM Application →Model output constrained or validated against a machine-readable schema so application code can consume fields without parsing free-form prose.
Careful: Schema-valid output can still contain incorrect values. Structure is not factual verification.
Learn it: Structured Outputs: JSON, Schema Validation, Constrained Decoding →A loosely coordinated multi-agent pattern in which local agent decisions and message exchange produce system-level behavior. The term is used inconsistently, so the actual topology, state ownership, and termination rules must be specified.
Careful: Multiple named agents do not guarantee useful specialization or emergent coordination.
A provider-defined instruction message or configuration supplied by the application to establish behavior and constraints within that provider's instruction hierarchy.
Careful: Priority rules, message roles, persistence, and visibility differ across APIs. Check the current provider contract.
Learn it: Agent Instructions as Executable Constraints →The latency experienced by the slowest portion of requests, commonly summarized with a high percentile under a stated workload and time window.
Careful: Tail latency is not the single slowest request and has no meaning without the percentile, population, and measurement boundary.
Learn it: Inference Metrics — TTFT, TPOT, ITL, Goodput, P99 →A decoding parameter that rescales logits before a probability distribution is formed. Higher positive values usually flatten the distribution; lower positive values sharpen it.
Careful: A zero setting is often implemented as greedy decoding, but exact behavior and determinism depend on the provider, sampler, seed support, and serving system.
A typed array with a shape, data type, and device placement that frameworks use to represent inputs, parameters, activations, and gradients. Automatic-differentiation metadata is framework- and operation-dependent, not an inherent property of every tensor.
Partitioning tensor operations within a model layer across devices, with collective communication combining partial results during the layer computation.
Careful: Tensor parallelism splits work inside layers. Pipeline parallelism places different layer groups on different devices.
Learn it: Scaling: Distributed Training, FSDP, DeepSpeed →An explicit rule that ends or pauses an agent run when it succeeds, fails, exhausts a budget, reaches a safe boundary, or requires escalation.
Careful: A stop sequence ends text generation; a termination condition decides whether the task or workflow should stop.
The mechanism, specification, reference, invariant, or human judgment used to decide whether observed program behavior is correct.
Careful: The model that wrote the code should not be treated as an independent oracle merely because you ask it whether its own output is correct.
A documented account of protected assets, trust boundaries, potential adversaries, assumed capabilities, attack paths, impacts, and planned controls.
Careful: A threat model prioritizes plausible risks; it is not a checklist that proves the system secure or predicts every future attack.
For one request with `N > 1` output tokens, the average post-first-token interval: `(t_N - t_1) / (N - 1)`. System distributions then aggregate those per-request averages.
Careful: TPOT is a per-request average. An individual inter-token latency is one gap between consecutive tokens, while time to first token includes the wait before output starts.
Learn it: Inference Metrics — TTFT, TPOT, ITL, Goodput, P99 →The elapsed time from submitting a generation request until the client receives the first output token or content event under a defined measurement boundary.
Careful: TTFT is not tokens per second. One measures startup latency; the other measures generation throughput after output begins.
An integer identifier produced by a model-specific tokenizer from text, bytes, images, audio, or another input representation. A token can be a whole word, part of a word, punctuation, whitespace, a byte sequence, or a special control symbol.
Careful: Character-to-token ratios vary by language, content, and tokenizer, so count with the target model's tokenizer or provider tools.
Learn it: Tokenizers: BPE, WordPiece, SentencePiece →An explicit allocation of token capacity across instructions, evidence, history, tool results, reasoning or working space, and output.
Careful: A token budget is a planning constraint. It is not the same as the model's maximum context window.
Learn it: Context Engineering: Windows, Budgets, Memory, and Retrieval →Converting an input representation into the ordered token identifiers a specific model or tokenizer accepts.
Careful: Tokenization is not always word splitting, and two models can assign different token counts and IDs to the same input.
A throughput measure reporting how many output tokens a serving system produces per unit time under a stated scope and workload.
Careful: TPS is not directly comparable across different tokenizers, workloads, quality settings, or measurement boundaries.
The complete agreement for a tool boundary: purpose, typed inputs, outputs, validation, permissions, side effects, errors, timeouts, idempotency, and evidence returned to the caller.
Careful: A JSON Schema is part of a tool contract, not the whole contract.
Learn it: Tool Use and Function Calling →A decoding method that restricts the next-token distribution to the k highest-scoring candidates, renormalizes their probabilities, and samples from that set.
Careful: Top-k uses a fixed candidate count, while top-p uses a probability-mass threshold whose candidate count changes by step.
A correlated record of one request or task across model calls, retrieval, tools, state transitions, retries, approvals, and evaluations.
Careful: A trace should record operational evidence, not expose hidden model reasoning, secrets, or unredacted sensitive content.
Learn it: OpenTelemetry GenAI Semantic Conventions →Starting from representations or parameters learned on one data distribution or objective and adapting them for another. The transferable components and update strategy depend on architecture and task.
Careful: Transfer is not limited to later layers, and successful transfer is not guaranteed when source and target tasks differ sharply.
A neural-network architecture built from attention, position information, feed-forward sublayers, residual connections, and normalization. Encoder, decoder, and encoder-decoder variants use different masks and information flows.
Careful: Self-attention does not imply unrestricted all-to-all attention in every transformer.
Learn it: The Full Transformer — Encoder + Decoder →An interface where data, instructions, identity, or authority crosses between components or principals that operate under different trust assumptions.
Careful: A network boundary is only one kind of trust boundary. Untrusted document text entering a privileged agent context also crosses one.
Learn it: Capstone 82 — Jailbreak Taxonomy →A model or training setup has insufficient effective capacity, optimization, features, or training signal to capture useful patterns in the training data.
A latent-variable model trained with a reconstruction objective and a regularization term that keeps an approximate posterior close to a chosen prior. The reparameterization estimator allows gradients through stochastic latent sampling.
Careful: A VAE does not force every latent distribution to one fixed Gaussian; the exact prior and approximate posterior are modeling choices.
A storage and indexing system that supports nearest-neighbor queries over vector representations, often with metadata filtering, persistence, and approximate indexes.
Careful: A vector database stores and searches vectors. It does not create high-quality embeddings or guarantee relevant retrieval.
A control point that blocks progress until defined evidence satisfies a correctness or quality criterion.
Careful: Verification checks whether evidence meets criteria. Approval grants authority to proceed, even when the evidence is already known.
Learn it: Verification Gates →A vision architecture that represents an image as a sequence of patch embeddings with position information and processes that sequence with transformer encoder blocks.
Careful: ViT is an architecture family, not every transformer that accepts images, and its patches are not inherently semantic objects.
Learn it: Vision Transformers (ViT) →A model that learns relationships between, or jointly processes, visual and language representations for tasks such as retrieval, description, question answering, or grounded generation.
Careful: Accepting an image does not prove the model uses it correctly, and a VLM is not necessarily able to generate images.
Learn it: Vision-Language Models — The ViT-MLP-LLM Pattern →Connecting a language expression to spatial evidence in an image or video, such as a region, object, mask, or tracked entity.
Careful: Visual grounding identifies where the referenced evidence is. General image captioning can describe a scene without localizing each claim.
Learn it: Cross-Attention Fusion →The finite mapping between token identifiers and the units a tokenizer can emit, including ordinary, byte-level, and special control tokens.
Careful: A model vocabulary is not a dictionary of human words; many entries are fragments, bytes, whitespace patterns, or control symbols.
An initial training phase in which the learning rate rises from a smaller value toward the main schedule's target value.
Careful: Warmup is not required for every model and does not make an otherwise unsuitable learning rate safe.
A trainable coefficient in a model transformation. Weights are usually organized into tensors, and optimization adjusts them to reduce the training objective.
Careful: Not every parameter is called a weight; biases, embeddings, and normalization scales are parameters too.
An update rule that reduces selected parameter magnitudes over training, often by multiplying weights by a shrinkage factor separate from the gradient update.
Careful: Decoupled weight decay is equivalent to an L2 loss penalty for some simple optimizers, but not generally for adaptive optimizers such as Adam.
In Git, a working directory attached to a repository and branch or commit, with shared object storage but its own checked-out files and index.
Careful: A worktree isolates checked-out files, not every process, port, cache, database, or secret on the machine.
Learn it: The Workbench on a Real Repo →A security model that grants no implicit trust from network location or asset ownership and instead evaluates each access request against identity, device, resource, policy, and current context.
Careful: Zero trust does not mean trusting nothing or blocking all automation. It means making trust decisions explicit, scoped, and continuously verifiable.
Learn it: Security — Secrets, API Key Rotation, Audit Logs, Guardrails →Performing a task from instructions or task framing without including task-specific demonstrations in the immediate input.
Careful: Zero-shot does not mean the model had no relevant pretraining, instruction tuning, tools, or retrieved context.