CCA Developer · M1 — MSO Foundations

1.1 Tokens

Everything Claude processes is tokens, not characters or words. Understanding tokens is the foundation for reasoning about cost, speed, and context limits.

What a token is. Claude uses a BPE (Byte Pair Encoding) tokeniser. Common English words are usually one token. Uncommon words, code symbols, and non-Latin scripts are split across multiple tokens.

Counting tokens programmatically. Use the SDK's token counter before sending expensive requests:

import anthropic

client = anthropic.Anthropic()

response = client.messages.count_tokens(
    model="claude-sonnet-5",
    system="You are a helpful assistant.",
    messages=[{"role": "user", "content": "Explain quantum entanglement."}]
)

print(response.input_tokens)  # e.g. 21

1.2 Context Window

The context window is the total number of tokens Claude can hold in one conversation — input and output combined. It is a shared resource.

The key constraint. When you call the API, you specify max_tokens for the output. The model will generate at most that many tokens. The input (system prompt + conversation history) plus the output must fit in the context window:

input_tokens + max_tokens ≤ context_window_size

If your input is 180K tokens and you request max_tokens=32768, you need a 212K+ context window.

KV cache and fill order. The model maintains a Key-Value (KV) cache of previously computed token representations. This is what makes multi-turn conversations efficient — tokens that were already processed don't need to be recomputed. However, the cache is bounded by the context window. When the conversation grows past the limit, you must decide what to drop.

Fill order strategy. Place content in this order for maximum cache efficiency:

1. System prompt (stable — cache this first)
2. Static documents or reference material
3. Conversation history (older turns first)
4. Current user message (dynamic — at the end)

Stable content at the front gets cached and reused. Dynamic content at the end avoids invalidating the cache.

1.3 Sampling and Non-Determinism

Claude generates tokens probabilistically. The same prompt will produce different outputs on different runs — this is non-determinism, and it is fundamental to how the model works.

Sampling parameters.

Temperature is NOT the only source of non-determinism. Even with temperature=0, outputs can vary because:

• Hardware floating-point differences across server instances
• Model weight updates (new model versions change behaviour)
• Token budget pressure (a model close to its max_tokens limit may abbreviate)
• Prompt caching introduces subtle timing differences

Designing for non-determinism. Don't fight it — design around it:

• Define expected output properties in your eval, not exact strings
• Use structured output (JSON schema or tool use) to constrain format
• Test across multiple samples to catch edge-case behaviours

1.4 The Claude Model Family

As of 2026, the Claude model family has four tiers:

Model selection rule. Start with Sonnet 5. Move to Opus only if Sonnet fails your eval benchmarks on the specific task. Move to Haiku only if Sonnet passes but your cost/latency budget is not met.

Accessing models via API. Models are identified by their ID string. There are two kinds of IDs:

• Alias: claude-sonnet-5 — always resolves to the latest stable Sonnet 5 release. Automatic updates; use for development.
• Pinned: claude-sonnet-5-20251101 — resolves to a specific release. No surprise changes; required when you have a validated eval suite. See M5 for detailed versioning strategy.

1.5 Reasoning Modes

Claude has two reasoning modes: basic (default) and extended thinking.

Basic mode. The model produces a direct response. Reasoning happens implicitly in the generation process. Fast, suitable for the vast majority of tasks.

Extended thinking. The model generates a scratchpad of reasoning tokens (visible to you as a thinking block in the response) before producing the final answer. This significantly improves accuracy on:

• Multi-step math and logic problems
• Complex code review that requires holding many constraints simultaneously
• Long-horizon planning tasks

response = client.messages.create(
    model="claude-opus-5",
    max_tokens=16000,
    thinking={
        "type": "enabled",
        "budget_tokens": 10000   # max tokens for the thinking block
    },
    messages=[{"role": "user", "content": "Prove that sqrt(2) is irrational."}]
)

for block in response.content:
    if block.type == "thinking":
        print("Thinking:", block.thinking)
    elif block.type == "text":
        print("Answer:", block.text)

When to use extended thinking.

1.6 Prompting Modes

Three prompting modes form a progression of specificity:

Zero-shot. Instructions only, no examples. Works well when the task is clearly specified and the model's training already covers the pattern.

{"role": "user", "content": "Classify this review as positive, neutral, or negative: 'The delivery was late but the product is excellent.'"}

Few-shot (one-shot / multi-shot). Instructions plus one or more demonstrations. Use when:

• Output format matters and words alone don't fully specify it
• The task requires a style or reasoning pattern the model needs to see

system = """Classify product reviews.

Example:
Review: "Broke after a week, terrible quality."
Classification: negative

Example:
Review: "Decent for the price, but nothing special."
Classification: neutral
"""

Guidelines for few-shot examples:

• Quality over quantity: three excellent, diverse examples beat ten mediocre ones
• Cover the hard cases: easy examples add no information; edge cases teach the boundary
• Match the format exactly: if production input has typos, include a typo in an example
• Keep examples adjacent to the instruction they illustrate

Multi-turn. Conversation history passed as the messages array. The model has access to the full prior exchange. Use for interactive assistants; be deliberate about how much history you retain (context management — see M2).

1.7 SDK vs REST vs Batch

When to use the official SDK.

The Python and TypeScript SDKs add:

• Automatic retry with exponential backoff (rate limit errors handled for you)
• Streaming response parsing (event-by-event without manual SSE parsing)
• Type-safe request and response objects
• Helper utilities (token counting, tool schema validation)

Use the SDK for all production integrations. Use raw REST only when the SDK is unavailable (unusual runtime, edge function with size limits).

# SDK — retries, types, streaming built in
client = anthropic.Anthropic(api_key=os.environ["ANTHROPIC_API_KEY"])

Synchronous vs streaming vs batch — choose based on the user experience.

Streaming example (Python SDK):

with client.messages.stream(
    model="claude-sonnet-5",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Summarise this report: ..."}]
) as stream:
    for text in stream.text_stream:
        print(text, end="", flush=True)

Batch example — submit 500 documents:

import anthropic

client = anthropic.Anthropic()

requests = [
    {
        "custom_id": f"doc-{i}",
        "params": {
            "model": "claude-haiku-4-5",
            "max_tokens": 256,
            "messages": [{"role": "user", "content": f"Summarise: {documents[i]}"}]
        }
    }
    for i in range(500)
]

batch = client.messages.batches.create(requests=requests)
print(batch.id)  # poll this ID to get results when ready

M1 Checkpoint

Key facts to remember from this module:

• 1 English word ≈ 1.3 tokens; non-English and code are more expensive
• Context = input + output combined — always budget both
• temperature=0 ≠ fully deterministic — hardware drift and model updates still cause variation
• Start with Sonnet 5 — move up/down based on eval results, not intuition
• Extended thinking costs output tokens — enable it only when basic mode fails your eval
• Streaming is default for user-facing output; Batches API for bulk non-urgent work