Resume Agent — from requirement to shipped, certified system

Grounding - no hallucination

Challenge: Early answers confidently invented resume facts. A resume assistant that makes things up is worse than none.

• • Tool-first hardened system prompt: every factual claim must trace to knowledge.yaml
• • Made "I don't have that" a first-class answer instead of a guess
• • Output guardrails: prompt-leak + off-topic checks before the reply is sent
• • Markdown link allow-list - only URLs present in the KB survive

Result: Grounded answers that pass the certification gate; unknowns are admitted, not fabricated.

Throughput - Groq free-tier TPM

Challenge: Each question makes up to 2 LLM calls (tool-pick, then synthesize) and re-sends the system prompt + tool schemas every time. Tokens/minute - not requests - was the binding limit, and a 429 surfaces to users as a "busy" reply that reads like a broken toy.

• • Consolidated to 2 lean tools (search_projects, get_profile) to cut tool-schema tokens re-sent on every call
• • RESPONSE_MODE caps: accuracy (full context) vs lean (trimmed history / tool-output / token budget)
• • First-turn response cache serves repeat + starter-chip questions with zero LLM tokens
• • One retry honoring Retry-After on a provider 429 before the busy message
• • Provider fallback chain (Groq -> Google -> xAI) so one 429 hops instead of failing

Result: A portfolio's worth of traffic stays comfortably inside the free tier.

Streaming UX

Challenge: Streaming intermediate agent passes leaked "let me look that up" narration to the user, and raw token chunks broke markdown and newlines mid-render.

• • ainvoke the final answer, then chunk it for the stream (ADR-3) - no intermediate narration
• • JSON-encoded SSE chunks so newlines and formatting survive the wire
• • Deferred markdown render during streaming to avoid half-parsed HTML
• • Follow-up suggestion chips parsed from the reply to guide the next question

Result: A stream that feels fast and alive while keeping a clean, single voice.

Tool grounding on small models

Challenge: Weak or local models mis-picked the section argument on a single "get_profile(section)" tool, returning the wrong slice of the KB.

• • TOOL_MODE=meta: expanded 7 dedicated zero-arg tools for Ollama (the tool name carries intent)
• • Lean 2-tool set for capable hosted models (fewest schema tokens)
• • Token-scored project search (2+ token or name match) to avoid flooding the KB
• • Tool output capped at 800 chars

Result: Reliable routing across providers - from a local 7B to hosted Groq/Gemini.

Injection guard tuning

Challenge: The base64-ish "80+ char" regex blocked legitimate long tokens (false positives), and regex alone is bypassable by paraphrase or homoglyphs.

• • Tightened the base64 rule to 120+ chars mixing case and digits
• • Centralized every guardrail/injection/PII pattern into guards.py with unit tests
• • Kept output-side guards (prompt-leak, off-topic, tool-call leak, link allow-list) as the real safety net

Result: Fewer false blocks on real input, with security rules now testable in isolation.

Timing-safe secret

Challenge: The X-Proxy-Secret check used ==, which short-circuits on the first differing byte and leaks length/prefix via timing - and that same secret gates whether X-Forwarded-For (rate-limit IP) is trusted.

• • Switched to hmac.compare_digest after a cheap presence check
• • Forwarded IP is only trusted when the secret matches

Result: A constant-time gate on the security-sensitive trust boundary.

Cold starts (free Render)

Challenge: The free backend spins down after ~15 min idle, so the next visitor paid a 30-50s cold start that reads as "broken".

• • A Cloudflare Worker cron pings /health every 10 minutes to keep the instance warm
• • An in-widget "starting up" fallback covers the rare genuine cold start

Result: Visitors hit a warm backend; the cold path degrades gracefully instead of hanging.

Config drift + reproducibility

Challenge: The Groq default model disagreed between the startup log and the provider factory, and ad-hoc os.getenv parsing threw at first call instead of failing clearly at boot.

• • Single pydantic-settings Settings object - one source of truth, validated at startup
• • Pinned requirements.lock for reproducible Docker builds
• • Added a test suite (33 tests) + ruff to the component

Result: No model-default drift, typed config, and a green lint/test gate.