| PLAIN/TEXT |
TRANSMISSION: DAILY ● |
|
| ISSUE 27/30 · COMPUTER SCIENCE |
~4 MIN |
NOW / AI CONTEXT · WEEK 05 — FROM RULES TO EMERGENCE
Context is a budget, not a cupboard
START HERE
Before an AI is released, training adjusts its internal numbers using many examples so it learns broad language patterns. Your current request then supplies the specific information for this answer. The request-time information is called context. It can include your question, instructions, earlier messages, documents, and results returned by tools. Adding more text can supply missing evidence, but it can also add irrelevant details, conflicting instructions, and extra computation.
|
/ WHY NOW
When an AI misses information, the tempting fix is to paste in every document, log and tool description. Current systems are also being engineered to avoid bloated inputs, retrieve what is needed, and reuse unchanged beginnings. More context can contain more evidence—but also more computation, distraction and conflicting instructions.
|
/ THE IDEA
Models split context into small text pieces called tokens. An attention layer gives each token scores that control how strongly information from other permitted tokens contributes while processing it. In a causal language model, each token normally uses itself and earlier tokens. That produces n(n+1)/2 permitted relationships—about half of n squared—so doubling context still creates about four times as many relationships. Modern systems use many optimisations and architectures, but the core lesson survives: unused context is not free. Good retrieval raises information density by selecting the small subset likely to matter for the current step.
THE FORMAL IDEA
causal attention relationships = n(n + 1) ÷ 2 = O(n²)
| n = number of tokens in the context | | each token uses itself and the tokens before it | | O(n²) means the relationship count grows quadratically; double n → about four times as many |
|
RUN THE TINY EXAMPLE
Double a prompt
10,000 tokens → about 50 million permitted relationships 20,000 tokens → about 200 million permitted relationships Retrieve 2,000 relevant tokens → less work and a cleaner evidence set
|
These are conceptual full-attention counts, not a provider’s latency promise. They show why doubling input can be more than a simple doubling inside a model.
/ SO WHAT?
Treat prompt construction like briefing a colleague. Include the governing instructions and evidence needed for this decision; link or retrieve the rest when it becomes relevant. Organisation can beat volume.
ONE CAVEAT |
| Not every model uses full quadratic attention across its entire context, and removing information can cause its own failures. Retrieval quality and preserving critical global instructions matter. |
KEEP THIS
Useful context is the smallest complete evidence set, not the largest pile that fits.
|
NEXT: The radius of no return
|
|
HARD IDEAS. PLAIN TEXT.
You are receiving this because you asked for one clear computer-science or physics idea each day.
UNSUBSCRIBE · VIEW IN BROWSER
Plain/Text / Brickston Consulting · 30 N Gould St Ste N, Sheridan, WY 82801 US · knowledge@plaintext.click
|
|