AI infrastructure for solo founders: scaling agents on a budget with serverless and caching

Most AI infrastructure advice assumes you have a DevOps team, a $10k/month cloud budget, and the appetite to run Kubernetes clusters. AI infrastructure for solo founders looks nothing like that — and the gap between enterprise architecture guides and what actually works when you’re shipping solo is wider than most tutorials acknowledge. You’re not Netflix. You don’t need to engineer for Netflix-scale problems. But you do need something that doesn’t collapse the moment you get a spike of real users, and that won’t drain your runway before you’ve validated anything.

This is a breakdown of the architecture decisions I’d make — and have made — running Claude-based agents as a solo operator trying to stay under $1,000/month total infrastructure cost while still shipping production-quality workflows.

The Misconceptions That Cost Solo Founders Real Money

Before getting into architecture, let’s clear up three things the tutorials get wrong.

Misconception 1: You need always-on compute. You don’t. Serverless functions handle the vast majority of agent workloads better than a $40/month VPS running at 3% CPU utilization. The exception is streaming responses with tight latency requirements — but if you’re building async workflows (document processing, lead qualification, content pipelines), serverless is strictly better and cheaper.

Misconception 2: You need to start with Sonnet or Opus. Claude Haiku 3.5 costs $0.80/MTok input and $4/MTok output. Sonnet 3.5 is $3/MTok and $15/MTok. For classification, extraction, and routing tasks — which make up maybe 60% of agent work — Haiku is indistinguishable in quality and costs roughly 4x less. I’ve run side-by-side comparisons against competing small models and Haiku holds up surprisingly well for structured tasks. Default to Haiku, escalate to Sonnet only when you can measure the quality difference mattering.

Misconception 3: Prompt caching is a nice-to-have. It’s not. It’s the single highest-ROI optimization available to solo founders. Anthropic’s prompt caching charges $0.30/MTok for cache writes and $0.03/MTok for cache hits on Haiku — a 90% reduction on cached tokens. If you have a 2,000-token system prompt that you’re sending on every request, caching it alone can cut your input token costs by 70%+ on high-volume workloads.

The Architecture: Serverless + Caching + Cheap Storage

Here’s the stack I’d recommend for a solo founder running multiple Claude agents in production:

• Compute: AWS Lambda or Cloudflare Workers (Workers wins on cold start latency, Lambda wins on ecosystem)
• Queue: AWS SQS or Cloudflare Queues for async agent tasks
• Cache: Redis via Upstash (serverless Redis, pay-per-request, no idle cost)
• Storage: S3 for documents and agent outputs
• Database: PlanetScale or Supabase free tier for structured data
• Observability: Helicone free tier for LLM call tracking

Total baseline infrastructure cost before LLM calls: roughly $15-30/month if you stay within free tiers aggressively. That leaves most of your $1k budget for actual API calls.

Implementing Prompt Caching Correctly

The Anthropic SDK makes prompt caching straightforward, but the placement of cache breakpoints matters. Here’s a working pattern for a document processing agent:

import anthropic

client = anthropic.Anthropic()

# System prompt is large and static — cache it aggressively
SYSTEM_PROMPT = """You are a document extraction specialist. Your job is to...
[2000+ token detailed instructions and examples]
"""

def process_document(document_text: str, extraction_schema: dict) -> dict:
    response = client.messages.create(
        model="claude-haiku-3-5",
        max_tokens=1024,
        system=[
            {
                "type": "text",
                "text": SYSTEM_PROMPT,
                "cache_control": {"type": "ephemeral"}  # Cache this block
            }
        ],
        messages=[
            {
                "role": "user",
                "content": [
                    {
                        "type": "text",
                        # If you have static reference docs, cache those too
                        "text": f"Schema to extract: {extraction_schema}",
                        "cache_control": {"type": "ephemeral"}
                    },
                    {
                        "type": "text",
                        # Dynamic content goes last — not cached
                        "text": f"Document to process:\n\n{document_text}"
                    }
                ]
            }
        ]
    )
    
    # Track cache performance in your metrics
    usage = response.usage
    cache_hit = getattr(usage, 'cache_read_input_tokens', 0)
    cache_miss = getattr(usage, 'cache_creation_input_tokens', 0)
    
    return {
        "result": response.content[0].text,
        "cache_hit_tokens": cache_hit,
        "cache_miss_tokens": cache_miss
    }

The key insight: cache breakpoints work from top to bottom, and everything after a non-cached block won’t be cached. Put your static content — system prompts, reference documents, extraction schemas — before your dynamic user content.

Semantic Deduplication with Redis

Beyond prompt caching (which Anthropic handles on their side), you should implement your own semantic cache. When users ask similar questions, don’t burn tokens on near-duplicate LLM calls. A simple approach using Upstash Redis with vector similarity:

import hashlib
import json
from upstash_redis import Redis

redis = Redis.from_env()

def get_cache_key(prompt: str, model: str) -> str:
    """Exact match cache for identical prompts."""
    content = f"{model}:{prompt}"
    return f"llm_cache:{hashlib.sha256(content.encode()).hexdigest()}"

def cached_claude_call(prompt: str, system: str, model: str = "claude-haiku-3-5") -> str:
    cache_key = get_cache_key(f"{system}:{prompt}", model)
    
    # Check cache first
    cached = redis.get(cache_key)
    if cached:
        return json.loads(cached)["response"]
    
    # Make the actual API call
    response = client.messages.create(
        model=model,
        max_tokens=1024,
        system=system,
        messages=[{"role": "user", "content": prompt}]
    )
    
    result = response.content[0].text
    
    # Cache for 1 hour — adjust TTL based on how time-sensitive your data is
    redis.setex(cache_key, 3600, json.dumps({"response": result}))
    
    return result

Exact-match caching is the highest-fidelity version of this. For semantic similarity caching, you’d need to embed the prompt and do a nearest-neighbor lookup — that’s worth implementing if you’re seeing high repetition in user queries. The semantic search implementation guide covers the embedding and retrieval patterns in detail.

Serverless Deployment: Lambda vs Workers for Agent Workloads

The choice between AWS Lambda and Cloudflare Workers comes down to a few concrete factors:

• Lambda: 15-minute max execution time, better for long-running document processing pipelines, native SQS integration, Python first-class support. Cold starts: 200-800ms with Python + dependencies.
• Workers: 30-second CPU limit (real-time clock is different), sub-5ms cold starts, runs at edge (lower latency globally), JavaScript/TypeScript only unless you use Workers AI. Better for low-latency response streaming.

For most Claude agent workloads — document processing, lead qualification, content generation — I’d choose Lambda because the 15-minute timeout gives you room to handle large documents without architectural gymnastics. For anything user-facing with real-time streaming, Workers is worth the JavaScript tax.

Handling the Long-Tail: Async Queues

The pattern that breaks production most often for solo founders: synchronous API calls on tasks that can take 10-30 seconds. Your frontend times out, your user thinks it’s broken, and you get support tickets.

The fix is a two-Lambda pattern: one Lambda accepts the job and returns a job ID immediately, a second Lambda does the actual processing triggered via SQS. Your frontend polls for the result.

# Lambda 1: Accept job (fast, returns immediately)
import boto3
import uuid

sqs = boto3.client('sqs')

def submit_job(event, context):
    job_id = str(uuid.uuid4())
    
    sqs.send_message(
        QueueUrl=os.environ['PROCESSING_QUEUE_URL'],
        MessageBody=json.dumps({
            "job_id": job_id,
            "document": event['body']['document'],
            "callback_url": event['body'].get('callback_url')
        })
    )
    
    return {"statusCode": 202, "body": json.dumps({"job_id": job_id})}

# Lambda 2: Process job (triggered by SQS, can take minutes)
def process_job(event, context):
    for record in event['Records']:
        payload = json.loads(record['body'])
        result = process_document(payload['document'])
        
        # Store result in S3 or DynamoDB
        store_result(payload['job_id'], result)
        
        # Optional: hit webhook when done
        if payload.get('callback_url'):
            notify_completion(payload['callback_url'], payload['job_id'], result)

This pattern also makes it trivial to add retry logic and fallback handling — SQS has built-in retry with exponential backoff and dead letter queues for jobs that consistently fail.

Real Cost Numbers: A Working Example

Here’s what a realistic solo founder workload looks like at $800/month total. Assume you’re running a document intelligence product with three agent types:

• Extraction agent (Haiku): Processes 5,000 documents/month, avg 800 input tokens + 200 output tokens per doc. With 70% cache hit rate on the 1,500-token system prompt: ~$18/month in API costs.
• Summarization agent (Haiku): 2,000 long-form documents, avg 3,000 input + 800 output. No good caching opportunity (docs are all different). ~$31/month.
• Reasoning/QA agent (Sonnet 3.5): 500 complex queries/month requiring better quality, avg 2,000 input + 1,000 output. ~$13.50/month.

Total LLM costs: ~$62.50/month. Infrastructure (Lambda, SQS, Upstash, S3, Supabase): ~$25/month. Monitoring via Helicone (free tier covers 10k requests): $0. Total: roughly $90/month — nowhere near the $1k ceiling, leaving you room to scale 10x before hitting budget constraints.

The math changes fast if you start using Sonnet for everything or skip caching. Running those same 5,000 extraction documents through Sonnet without caching costs ~$155/month just for that one agent. Model selection and caching are not premature optimizations — they’re the difference between a sustainable product and one that kills your runway.

Observability Without Enterprise Pricing

You cannot optimize what you can’t see. At minimum, track per-request token counts, model, latency, and cache hit ratio. The Helicone vs LangSmith vs Langfuse comparison covers the full tradeoff — my short take for solo founders: Helicone’s free tier (10k requests/month) is the least friction to get started, and their proxy-based integration means you change one line of code:

# Change just the base_url — everything else stays identical
client = anthropic.Anthropic(
    base_url="https://anthropic.helicone.ai",
    default_headers={
        "Helicone-Auth": f"Bearer {os.environ['HELICONE_API_KEY']}",
        "Helicone-Cache-Enabled": "true",  # Helicone-level caching on top
    }
)

Once you exceed Helicone’s free tier, Langfuse self-hosted on a small EC2 instance is ~$5/month and gives you the full platform.

When to Add Complexity

The architecture above — serverless functions, Redis cache, SQS queue — handles most solo founder use cases through $20k MRR comfortably. Here’s what triggers a real upgrade:

• Sub-100ms latency requirements: Move to Workers or a persistent FastAPI service on a small instance. Serverless cold starts won’t cut it.
• Concurrent streaming to many users: Lambda’s concurrency model gets expensive. A persistent service on a $20/month Fly.io machine handles streaming better.
• Complex multi-agent orchestration: At some point you’ll want the Claude Agent SDK over raw API calls — it handles tool use, conversation state, and agent loops with much less boilerplate than rolling your own.
• Persistent agent memory: SQLite on Lambda doesn’t work (ephemeral filesystem). You’ll need a proper persistent memory layer — which is a different architectural problem.

Don’t add these prematurely. The failure mode I see most often with solo founders is over-engineering the infrastructure before validating the product. Ship on the simple stack, then pay the refactoring cost when you have real users and real load numbers to optimize against.

The Bottom Line for AI Infrastructure Solo Founders

If you’re just starting out: Deploy on Lambda with Upstash Redis for caching, default to Haiku with prompt caching enabled, and add Helicone for observability. You’ll be under $100/month and have room to scale 10x before rethinking anything. The architecture in this article runs reliably at that budget.

If you’re at $5k-20k MRR: The same stack still works. Start splitting agents by task type (Haiku for extraction/routing, Sonnet for reasoning), implement semantic caching on your most repetitive query patterns, and add a dead-letter queue for failed jobs. Observability becomes critical here — you need to know which agents are eating budget.

If you’re considering a persistent server instead of serverless: Only make that move when you have measured evidence that cold starts or timeout limits are actually hurting you. The operational overhead of a persistent service (monitoring uptime, handling crashes, managing deployments) is real work that serverless offloads entirely. For a solo founder, your time has a real cost too.

The goal of lean AI infrastructure for solo founders isn’t to run the cheapest possible system — it’s to run a system that’s cheap enough to validate your product, reliable enough to not embarrass you with users, and simple enough that you can maintain it without a team. This architecture does all three.