On this page Overview
KB Playground — build, tune, share, and degrade gracefully
An extension of the Resume Agent into a self-serve playground: draft KB sessions with live pipeline traces, publish-to-share with passkey auth, triple-gate quotas, and a two-phase TTL that degrades from live LLM to cached answers. Built to teach agent concepts interactively while staying inside free-tier limits.
Try it now — open the KB Playground and build your own agent. Live pool counters appear in the builder UsageMeter; owner metrics require a stored key and are not linked publicly.
See it in action
The playground is the demo. Load an illustrative sample profile based on famous public figures (not real user data), tune LLM params, watch pipeline traces in the builder, then publish a share link and experience the visitor flow.
1. Builder
Edit KB via sample profile picker or from scratch, pick provider/temperature, chat with full trace inspector and flow diagram in the expandable workspace.
2. Publish
Click Share → get slug + passkey receipt. KB stored in Supabase for 7 days.
3. Share link
Open /playground/{slug} as a visitor. Live LLM for 48h, then cache-only with suggestion chips.
The problem & my role
Problem
The resume agent proves grounded Q&A works, but visitors cannot see how it is tuned, quota-gated, or debugged. Agent skills are invisible behind a chat widget.
My role
BSA who scoped dual-mode flows, TTL phases, and quota tiers; engineer who built the full stack across SvelteKit, Cloudflare, FastAPI, and Supabase.
Thesis
A public demo must degrade gracefully. Live LLM → cached answers → expired link — each phase bounded by hard quotas so free-tier budget survives real traffic.
Epic — Features — Stories
Click a feature to see user stories, reasoning, and MVP placement.
"As a visitor exploring AI agent concepts on my portfolio, I want to build and tune a knowledge-base assistant interactively — then share a time-bound demo link — so I can learn agent tuning, quota management, and debugging without running my own infrastructure."
F1 - Draft preview & KB editing
Why it exists: Before anyone publishes, they need a safe sandbox: edit KB via form or YAML, tune LLM params, and chat against an in-memory session with zero database writes.
MVP call: MVP. Shipped first — the core builder experience.
- • As a builder, I want to edit KB in a form or raw YAML so I can iterate quickly.
- • As a builder, I want to load one of 8 illustrative sample profiles (public figures, demo disclaimers) to start fast.
- • As a builder, I want a 30-min RAM session so I can tune without persisting data.
- • As a builder, I want to pick provider, temperature, and tool mode for my draft.
MVP-first delivery
Sequenced by dependency: builder before publish, quotas before scale, traces last.
Phase 0 - Builder
"Prove the split-screen KB editor + draft chat loop."
- • Form + YAML editor with form↔YAML sync
- • 8 illustrative sample profiles (public figures, explicit disclaimers)
- • 30-min in-memory session (SessionStore)
- • Turnstile on session create
- • Basic SSE chat against dynamic KB
Phase 1 - Publish & share
"Let builders share time-bound demo links."
- • Supabase RPC create/update with passkey
- • 8-char slug + one-time receipt modal
- • Shared-link visitor chat UI
- • Owner passkey edit flow
- • Owner delete flow (passkey + slug confirm; releases same-day publish quota slot)
Phase 2 - Quotas & cache
"Stay inside free-tier LLM budget at scale."
- • Edge KV tiered daily caps (builder vs share pools)
- • Two-phase TTL: 7-day link, 48h live LLM, then cache-only
- • First-turn answer cache in Supabase
- • Cache suggestion chips + quota dashboard
Phase 3 - Trace debugging
"Make agent tuning visible and teachable."
- • step SSE events via TraceCollector (latency_ms + redacted payloads)
- • Browser-only traceStore + TraceInspector
- • DynamicFlowTrace XYFlow with bidirectional trace↔flow linking
- • Expandable PlaygroundWorkspace modal (8 layout presets)
- • Config fingerprint lock on runtime param change
Dual-mode architecture
Draft and shared paths share the same UI shell but diverge at storage, TTL, traces, and quota pools.
| Dimension | Draft preview | Shared link |
|---|---|---|
| Storage | In-memory SessionStore (Render RAM) | Supabase playgrounds table via RPC |
| TTL | 30 minutes from session create | 7-day link; 48h live LLM window; then cache-only |
| LLM params | Builder-chosen provider, temperature, tool mode | Site defaults only (not draft runtime params) |
| Pipeline traces | Full step SSE + browser traceStore | Simple meta → message stream only |
| Quota pool | Builder pool (2000/day global, 24/IP) | Share pool (100/day) + per-slug + per-IP caps |
| Answer cache | None — every turn hits LLM | First-turn cache in playground_answer_cache |
FastAPI Gateway
Gate 2 — BackendSanitizes input, enforces injection guards, routes draft vs slug chat, and manages session store.
- • Input sanitization with length cap
- • Injection guard before LLM; grounding check for playground KB
- • 8-turn session cap; slug phase detection (live vs cache_only)
Triple-gate quota management
Edge burst → KV daily caps → FastAPI turn limits → provider RPM/RPD pools. Cache hits bypass LLM quota consumption.
| Gate | Layer | Limit | Scope |
|---|---|---|---|
| Gate 1 — Edge burst | Cloudflare Pages Function | 5 req/min per IP | Per-IP in-memory bucket |
| Gate 1 — Builder pool | Cloudflare KV | 2000/day global | Site-wide builder traffic |
| Gate 1 — Builder per-IP | Cloudflare KV | 24/day per IP | Per-visitor builder traffic |
| Gate 1 — Share pool | Cloudflare KV | 100/day global | Site-wide shared-link traffic |
| Gate 1 — Per-IP share | Cloudflare KV | 8/day per IP | Per-visitor shared-link traffic |
| Gate 1 — Per-slug daily | Cloudflare KV | 24/day per slug | Per published slug |
| Gate 1 — Turns per visitor | Cloudflare KV | 8/day per IP per slug | Per visitor per slug |
| Gate 1 — Publish per IP | Cloudflare KV | 2/day per IP | New shareable links per IP |
| Gate 2 — Session turns | FastAPI | 8 turns per conversation | In-session turn cap |
| Gate 3 — Provider pool | LLM registry | Primary Gemini Flash-Lite 15 RPM / 500 RPD + ordered fallbacks (Groq, etc.) via LLM registry | Provider free-tier reserves |
Live pool counters appear in the embedded UsageMeter on the
playground builder (load once + Refresh; updates after each chat/publish via response headers, not polling). The owner metrics dashboard requires a stored metrics key — provider telemetry is not publicly accessible. Daily counters live in Cloudflare KV and reset at UTC midnight;
exact key schemes and RPC shapes are omitted from this public write-up.
Interactive trace debugging Builder-only
Draft mode emits step SSE events. Traces live in browser memory only — never
sent to Langfuse.
Edge proxy Request received at Cloudflare Pages Function; quota pre-check.
Gateway FastAPI route handler; session or slug resolution.
Input guard Injection pattern check; canned response on match.
Agent planning LangGraph entry; dynamic system prompt from KB YAML.
LLM round Provider call with tool schemas; may loop.
Tool execution KB search/get over structured YAML sections.
Grounding check Fails the turn if no KB content supports the answer.
Output guard Prompt-leak and off-topic checks before stream.
SSE stream Token chunks emitted; step events in builder mode only.
UI components
- PlaygroundWorkspace — expandable modal with 8 layout presets (builder, all-four, chat-focus, trace-focus, and solo-panel modes)
- SampleProfilePicker — 8 illustrative demo personas with explicit disclaimers (not real user data)
- TraceInspector + TraceStepDetail — step list with redacted I/O payloads (latency_ms per step)
- DynamicFlowTrace — XYFlow topology with bidirectional trace↔flow step highlighting
- TurnSelector — navigate traces by chat turn
- LlmRuntimeParams — provider, temperature, tool mode, simulate_429 retry demo
- traceStepSubtitle — compact step subtitles (token totals, tool_call names, completion char counts)
- Config fingerprint lock — blocks chat after runtime param change without reload
Supabase integration
playgrounds table
- • 8-char hex slug, bcrypt passkey hash, kb_yaml, expires_at
- • max_turns per slug (default 8), is_public flag
- • RPC-only create, update, delete, and get-public paths
- • Active-link cap enforced at publish (parameterized via runtime config; 30 in production)
playground_answer_cache
- • PK: slug + kb_version + question_norm
- • First-turn only; max 100 rows per slug
- • 16k char cap on cached answer rows (persist guard; live LLM output bounded by max_tokens ~2048)
- • Top cached questions power suggestion chips on shared links
All client access goes through FastAPI → PostgREST RPC. Anon direct table grants are revoked; passkey auth is app-layer (no Supabase Auth enrollment).
Security & request trimming
Risk register
- Quota exhaustion / cost spike — Triple-gate quotas + cache hits skip LLM bump + parameterized active link cap (30 in prod).
- Automated publish spam — Turnstile on KB load (session create); 2 publish attempts per IP per day (KV, UTC reset).
- Oversized KB payloads — 20 KB cap enforced at edge on session create and publish.
- Passkey loss — Explicit UX: no recovery path; passkey shown once on publish receipt.
- Prompt injection via custom KB — Same injection-guard patterns as resume agent; grounding check on playground KB.
- PII in user-typed messages — sanitize_input + trace redaction; honest privacy notice on playground page.
- Supabase anon key exposure — SECURITY DEFINER RPCs only; anon direct table grants revoked; passkey auth is app-layer (no Supabase Auth enrollment); service role never in frontend.
Input & context trimming
- Input sanitization — length cap, strip non-printable
- Injection guard — pattern checks before LLM
- Grounding check — fail if KB tools return empty
- History trim — system + last 6 turns (builder: 2–10 configurable)
- Turnstile — bot gate on KB load (session create; publish inherits prior gate)
- KB payload cap — 20,000 chars max at edge on session create and publish
- Owner delete — passkey-verified permanent removal; same-day delete may restore one publish attempt
Non-functional requirements
- Cost — $0 target — free-tier LLM pools (Gemini primary + Groq/etc. fallbacks) + Supabase free + Cloudflare Pages/KV free tier.
- Latency — SSE streaming; cache hits return in <100ms without LLM round-trip.
- Availability — keepwarm-worker cron + graceful cold-start UX on Render free tier.
- Privacy — Draft traces browser-only; no Langfuse for playground; PII redaction in trace display.
- Teachability — Step-level traces + flow diagram make agent pipeline visible to learners.
Challenges & tuning
Real pain points from building a cost-conscious public demo platform.
Free-tier LLM budget
Challenge: Primary Google Gemini Flash-Lite pool (15 RPM / 500 RPD) is tight — a single viral shared link could exhaust one provider.
- • Triple-gate quotas: edge KV daily caps, FastAPI turn limits, provider pool reserves
- • LLM registry ordered fallback chain (LLM_FALLBACKS, e.g. Groq) on 429/errors — builder and slug paths use use_fallbacks=True
- • Cache hits skip LLM quota bump on shared slug traffic
- • Separate builder (2000/day, 24/IP) and share (100/day) pools
Result: Predictable daily spend with live telemetry via the owner metrics dashboard.
Draft vs shared config drift
Challenge: Builders tune provider and temperature in draft, but shared links use site defaults. Silent mismatch would confuse visitors.
- • Runtime config fingerprint hashes LLM params with KB
- • Chat blocked if fingerprint changes without session reload
- • Help text on playground page explains draft vs shared behavior
Result: No silent behavior changes — builders know shared links use site defaults.
1-hour TTL too short
Challenge: Original spec used 1-hour link TTL. Demo links expired before recipients could try them.
- • Evolved to 7-day link TTL with separate 48-hour live LLM window
- • Cache-only phase keeps links useful after LLM window ends
- • KB edits do not extend expires_at (two-phase migration)
Result: Links usable for a week; LLM cost bounded to 48 hours from publish.
Cold starts on Render free tier
Challenge: Render free web services spin down after 15 min idle — first request pays 30–60s cold start.
- • keepwarm-worker Cloudflare cron pings GET /health every 10 min
- • Graceful loading states in chat UI during warm-up
Result: Sub-second responses after warm; cold path degrades gracefully.
Trace data sensitivity
Challenge: Pipeline traces contain system prompts, tool I/O, and user messages — sending to Langfuse would leak builder experiments.
- • observe=False on playground agent runs
- • traceStore stays browser-only (never persisted server-side)
- • Trace payloads redacted in the browser inspector
Result: Full debugging in builder without observability leakage.
KB edit invalidates cache
Challenge: Owner edits KB after visitors cached answers — stale responses would mislead.
- • kb_version = SHA-256 prefix of kb_yaml on cache rows
- • Passkey-verified KB update clears stale answer cache
- • Cache lookup keys include kb_version + normalized question
Result: Stale answers prevented after any KB update.
Key decisions (ADRs)
Dual-mode: RAM draft vs Supabase shared
Draft sessions need full trace debugging and custom LLM params without DB cost. Shared links need persistence and passkey auth — splitting storage models keeps each path simple.
Two-phase TTL (7d link / 48h live / cache-only)
A single TTL cannot balance "link stays shareable" with "LLM cost stays bounded." Separating link lifetime from inference window enables progressive degradation.
Edge KV as authoritative daily quota
FastAPI rate limiters are process-local and lost on Render restarts. Cloudflare KV counters survive deploys and work across edge instances — the right layer for daily caps.
Browser-only traces (no Langfuse for playground)
Builder experiments contain arbitrary KB content and tuned prompts. Langfuse is for the certified resume agent, not user-generated playground sessions.
Supabase RPC-only access (no direct table reads)
Anon key in the browser cannot safely expose playgrounds or answer_cache tables. SECURITY DEFINER RPCs with passkey verification keep writes gated and reads scoped.
First-turn cache only
Multi-turn cache keys explode combinatorially and stale faster. Caching turn 1 covers the common "try a starter question" pattern while keeping invalidation tractable.
Skills this exercised
Business analysis / product
- • Extended resume agent epic into playground features with clear MVP boundaries
- • Sliced publish, quota, cache, and debugging into phased delivery
- • Defined progressive degradation states (live → cache-only → expired)
- • Documented triple-gate quota model with measurable daily caps
- • ADR-style decisions for dual-mode storage and TTL phases
Engineering
- • LangGraph dynamic KB agent with TraceCollector step events
- • templated system prompts with injection-safe placeholders
- • Supabase RPC-only security with pgcrypto passkey hashing
- • Cloudflare KV tiered quota counters with cache-hit bypass
- • Svelte 5 runes: traceStore, PlaygroundWorkspace layouts, bidirectional trace↔flow diagrams
- • Two-phase TTL implementation with answer cache invalidation
Technology stack
Built on the Resume Agent foundation. Try the live playground or get in touch to discuss agent architecture.