Extended thinking vs chain-of-thought: when Claude’s reasoning modes actually matter

If you’ve run Claude extended thinking benchmarks against standard chain-of-thought prompting and found mixed results, you’re not alone. The Anthropic docs make extended thinking sound universally better — it’s not. Whether you should pay the latency and token cost for extended thinking or stick with a well-structured CoT prompt depends almost entirely on the task type and your tolerance for waiting 15–30 seconds for a response.

I’ve benchmarked both approaches across four representative agent tasks: multi-step code debugging, mathematical reasoning, ambiguous classification, and structured data extraction. The results are more nuanced than “extended thinking wins at hard problems.” Let me show you what the numbers actually look like and how to make the call at the architecture level.

What Extended Thinking and Chain-of-Thought Actually Do

These two approaches solve the same underlying problem — getting Claude to reason before committing to an answer — but they do it at fundamentally different layers.

Chain-of-Thought Prompting

CoT is a prompting technique. You instruct the model in your system or user prompt to “think step by step,” “reason through this before answering,” or show your working. The model reasons inside the visible response stream using its standard inference budget. No special API parameters needed, works on every Claude model, and you pay normal per-token rates on the output.

The reasoning is visible, which is useful for debugging and for grounding subsequent steps in agents. The downside: the model can shortcut its reasoning if the token budget feels tight, and it will sometimes narrate plausible-sounding steps rather than genuinely backtracking through alternatives. If you’re not already using structured prompt techniques alongside your CoT instructions, quality variance is high.

Extended Thinking (claude-3-5-sonnet + claude-3-7-sonnet)

Extended thinking is a model-level feature, not a prompting trick. You enable it via the API with thinking: {type: "enabled", budget_tokens: N}. The model spends up to N tokens on an internal scratchpad — this thinking block is returned in the response but billed separately and can’t be injected back into conversation history in subsequent turns without special handling.

The key difference: extended thinking uses a genuine search process. Claude 3.7 Sonnet with extended thinking will actually backtrack, reconsider, and explore dead ends in a way that visible CoT rarely does. You’re paying for real exploration, not narrated confidence.

import anthropic

client = anthropic.Anthropic()

# Extended thinking — costs more, takes longer, better on hard problems
response = client.messages.create(
    model="claude-3-7-sonnet-20250219",
    max_tokens=16000,
    thinking={
        "type": "enabled",
        "budget_tokens": 8000  # thinking tokens, billed separately
    },
    messages=[{
        "role": "user",
        "content": "Debug this recursive function that's producing incorrect fibonacci values..."
    }]
)

# Access thinking block and response separately
for block in response.content:
    if block.type == "thinking":
        print(f"[THINKING]: {block.thinking[:200]}...")  # usually don't show users this
    elif block.type == "text":
        print(f"[ANSWER]: {block.text}")

# Standard CoT — cheaper, faster, good enough for structured problems
response = client.messages.create(
    model="claude-3-5-sonnet-20241022",
    max_tokens=4000,
    system="Think through problems step by step before giving your final answer. Show your reasoning explicitly.",
    messages=[{
        "role": "user",
        "content": "Debug this recursive function that's producing incorrect fibonacci values..."
    }]
)

Benchmark Results: Four Real Agent Tasks

All tests used claude-3-7-sonnet-20250219 for extended thinking and claude-3-5-sonnet-20241022 for CoT (you can’t enable extended thinking on 3.5 Sonnet). Twenty runs per task, measuring accuracy, latency p50/p95, and approximate cost per call at current API pricing.

Task 1: Multi-Step Code Debugging

A Python function with three interconnected bugs — one syntax-adjacent, one logic error in a loop boundary, one incorrect variable scope. Extended thinking hit 17/20 correct root cause identifications. CoT with explicit step-by-step instructions hit 14/20. The gap came from cases where the first plausible bug explanation was wrong — extended thinking caught itself and kept looking, CoT tended to commit to the first reasonable story.

Cost difference matters here: extended thinking at 8k budget tokens ran ~$0.024 per call. CoT on 3.5 Sonnet ran ~$0.004 per call. Six times more expensive for a 15-percentage-point accuracy gain. Whether that’s worth it depends on whether downstream cost of a missed bug is higher than $0.02.

Task 2: Mathematical Reasoning (Multi-Step Word Problems)

This is where extended thinking earned its reputation. On problems requiring 4+ steps with intermediate unit conversions, extended thinking went 19/20. CoT went 12/20. The failure mode for CoT was consistent: correct individual steps but wrong propagation of intermediate results, particularly when a step required reformulating the problem rather than just executing the next operation.

If you’re building anything that involves financial calculations, logistics optimization, or scientific unit conversions, extended thinking is not optional for hard cases. The accuracy gap is too large to paper over with retry logic — you’d need to fire 3-4 CoT calls to match the reliability of one extended thinking call, which erodes the cost advantage entirely. This connects to a broader point about retry logic for production LLM pipelines — sometimes more retries don’t fix a fundamental reasoning ceiling.

Task 3: Ambiguous Classification

Classifying customer feedback into one of eight overlapping support categories where multiple labels could apply. Extended thinking and CoT were nearly identical: 15/20 and 14/20 respectively. Here, the bottleneck isn’t reasoning depth — it’s ambiguity in the label definitions themselves. No amount of internal scratchpad fixes that. This is the task type where you’re paying a 6x premium for essentially no gain.

Task 4: Structured Data Extraction

Extracting JSON from messy invoice text — varied formats, inconsistent field names, missing optional fields. Both approaches handled this well. CoT at 13/20 exact-match JSON, extended thinking at 14/20. Not a meaningful difference. For extraction tasks, your prompt structure and output schema definition matter far more than thinking depth. See the structured data extraction implementation guide for a deeper treatment of this.

Latency Reality Check

This is where the docs are optimistic. Extended thinking with an 8k budget adds 12–28 seconds to response time in my tests (p50: ~15s, p95: ~27s). That’s on top of the normal generation time. With a 4k budget you can get it down to 8–14s, but accuracy on the hardest problems drops noticeably.

For synchronous user-facing features, this is a hard constraint. For async agent pipelines where you’re already waiting on tool calls, file reads, or API responses, an extra 15 seconds is often irrelevant. The latency question is really a UX question, not a model quality question.

Head-to-Head Comparison

When Extended Thinking Is the Wrong Tool

The positioning of extended thinking as a general upgrade is misleading. There are specific cases where it actively hurts you:

• High-volume pipelines: If you’re processing thousands of documents daily, the cost multiplier kills margins. CoT on Haiku will outperform extended thinking on Sonnet on a cost-per-correct-answer basis for most classification and extraction work.
• Hallucination-prone tasks: Extended thinking doesn’t reduce factual hallucination — it’s a reasoning enhancement, not a knowledge enhancement. If your problem is Claude confidently stating wrong facts, longer reasoning about wrong premises doesn’t help. You need grounding strategies instead. Our guide on reducing LLM hallucinations in production covers what actually works.
• Simple routing and triage: Single-step decisions with clear criteria don’t need extended thinking. You’re burning compute for no gain.
• Real-time user interactions: 15-second response times are genuinely unusable in chat interfaces. Users interpret this as broken, not thoughtful.

Implementing a Hybrid Routing Pattern

The practical production pattern isn’t “pick one” — it’s routing tasks to the right mode based on detected complexity. Here’s a lightweight implementation:

import anthropic
from enum import Enum

class ReasoningMode(Enum):
    FAST = "fast"        # CoT on Haiku
    STANDARD = "standard"  # CoT on Sonnet
    DEEP = "deep"        # Extended thinking on 3.7 Sonnet

def classify_task_complexity(task: str, client: anthropic.Anthropic) -> ReasoningMode:
    """Quick cheap classification to route to the right reasoning mode."""
    response = client.messages.create(
        model="claude-haiku-20240307",  # ~$0.0003 for this call
        max_tokens=50,
        system="""Classify task complexity. Reply with only: FAST, STANDARD, or DEEP.
        FAST: extraction, simple classification, summarization, formatting
        STANDARD: multi-step reasoning, code review, structured analysis
        DEEP: mathematical proofs, complex debugging, multi-constraint optimization""",
        messages=[{"role": "user", "content": f"Task: {task[:500]}"}]
    )
    
    label = response.content[0].text.strip().upper()
    return ReasoningMode[label] if label in ReasoningMode.__members__ else ReasoningMode.STANDARD

def run_with_appropriate_reasoning(task: str, client: anthropic.Anthropic) -> str:
    mode = classify_task_complexity(task, client)
    
    if mode == ReasoningMode.FAST:
        response = client.messages.create(
            model="claude-haiku-20240307",
            max_tokens=2000,
            system="Think step by step before answering.",
            messages=[{"role": "user", "content": task}]
        )
        return response.content[0].text
    
    elif mode == ReasoningMode.STANDARD:
        response = client.messages.create(
            model="claude-3-5-sonnet-20241022",
            max_tokens=4000,
            system="Think through this carefully, showing your reasoning at each step.",
            messages=[{"role": "user", "content": task}]
        )
        return response.content[0].text
    
    else:  # DEEP
        response = client.messages.create(
            model="claude-3-7-sonnet-20250219",
            max_tokens=16000,
            thinking={"type": "enabled", "budget_tokens": 8000},
            messages=[{"role": "user", "content": task}]
        )
        # Return only the text response, not the thinking block
        return next(b.text for b in response.content if b.type == "text")

The routing call costs ~$0.0003 on Haiku and saves you the full extended thinking premium on tasks that don’t warrant it. At scale this matters: routing 80% of tasks to fast/standard mode and only 20% to deep mode brings your blended cost down from ~$0.025 to ~$0.008 per call while preserving accuracy where it counts.

This pattern works well in async agent pipelines. If you’re building agents with tool use, the async nature of tool calls gives you natural places to run extended thinking without blocking user-perceived latency. See Claude tool use with Python for how to structure these pipelines.

Token Budget Tuning for Extended Thinking

The budget_tokens parameter is worth spending time on. Anthropic’s docs suggest 8k–16k for hard problems, but in my testing:

• 1k–2k budget: Barely better than CoT. The model doesn’t have room to explore meaningfully.
• 4k–6k budget: Sweet spot for most “hard but not extreme” tasks. Latency ~10–15s, accuracy close to full budget.
• 8k–10k budget: Needed for genuine mathematical proofs and deep algorithmic debugging. Latency ~15–25s.
• 16k+ budget: Diminishing returns for most tasks. Save this for research-grade problems with many unknowns.

You can also set budget_tokens dynamically based on task metadata — question length, detected domain, presence of numbers or code blocks. A lightweight heuristic gets you 90% of the accuracy benefit of a full classifier.

The Verdict: Which Reasoning Mode to Use

Based on the Claude extended thinking benchmarks above, here’s how to make the call:

Choose extended thinking (Claude 3.7 Sonnet) if: your task involves multi-step mathematical reasoning, complex algorithmic debugging with non-obvious root causes, multi-constraint planning where the model needs to backtrack, or any problem where accuracy is significantly more valuable than latency and cost. The accuracy premium on hard problems is real and worth it.

Choose chain-of-thought prompting if: you’re doing classification, extraction, summarization, or any task where the reasoning ceiling isn’t the bottleneck. Also choose CoT when you need compatibility across model tiers (Haiku for volume, Sonnet for quality), when you’re serving real-time user requests, or when your pipeline runs thousands of calls per day and cost is a hard constraint.

For solo founders and small teams: default to CoT on Claude 3.5 Sonnet with good prompt structure. Add extended thinking selectively for the specific high-stakes decisions in your pipeline where wrong answers have real downstream cost. Don’t apply it uniformly — you’ll burn budget on tasks that don’t benefit.

For enterprise teams with accuracy SLAs: implement the routing pattern above. The classifier overhead is negligible compared to the savings from not running extended thinking on extraction and classification work. Reserve your deep reasoning budget for the 10–20% of tasks where it actually moves the needle.

The documentation oversells extended thinking as a universal improvement. It’s not. It’s a targeted tool for a specific class of reasoning-heavy problems, and knowing which class your task belongs to is the skill that actually matters here.