Ale Cabreradparents · 2026
Building a Source-First AI Agent for Traveling Families
ShippedTeam: Solo, Founder/PM10 weeks
TL;DR
- An AI search agent for worldschooling families that plans its own search, checks what it found, and only cites sources it can actually show.
- The LLMs were grounded on the original hand-curated dataset of 300 entries, so there was a real quality bar before anything was automated.
Outcome
- 20real users gave feedback, two product changes shipped from it
- 75%of 163 hand-rated results judged accurate (122), 36 partial, 3 needing sources, 2 hallucinated
- 1,330questions traced end to end across 291 sessions and 4 countries, testing included
- 5,668catalog entries live at dparents.com: 3,954 resources, 777 entities, 667 places, 270 regulations
- 81%of entities (626 of 777) and 100% of regulations (270) human-verified
- 72%of resources AI pre-screened (2,857 of 3,954), human review in progress
- →every source shown in an answer is a real search hit, never a model-invented URL
Saw It
- Worldschooling and traveling families make real-time decisions with incomplete information. The answers exist, buried in Facebook groups, WhatsApp threads, and creator videos.
- The gap is trustworthy delivery, not content. A wrong or made-up recommendation mid-trip costs a family time and trust.
- Named it the "Vasari Corridor problem": a family visits the Uffizi, misses a worthwhile detour 10 meters away, and only finds out afterward on a blog.
↓→
Figured It Out
- A single search query can't solve this. The agent has to decide where to look, judge if the results are good enough, and know when to dig further or stop.
- Trust had to be a data problem first: a structured taxonomy with review states, seeded by a hand-curated set, before any automation.
- One principle anchored every decision: show families where the answer came from.
↓→
Did It
- Shipped a LangGraph agent that plans, searches, evaluates, then refines or finalizes, across four tools: catalog content, catalog entities, web, and YouTube.
- Routed models by job: Haiku for planning and evaluating, Sonnet for the final answer.
- Traced every run end to end (1,330 questions: tools chosen, queries tried, outcome) and rated 163 results by hand: 75% accurate, 2 hallucinated.
My Role
- Defined the problem, the Vasari Corridor use case, and the trust principle the product is built on.
- Hand-curated the original dataset of 300 entries that the LLMs were grounded on.
- Designed the taxonomy (Places, Entities, Events, Regulations, Resources) and the review states behind it.
- Owned the guardrail spec: which failure modes to block (invented source URLs, unsourced numbers in regulation notes) and which to label for the user.
- Made the build-philosophy call: vibe-coded with Claude Code and Cursor, because the deliverable was product judgment and technical fluency in directing AI tools, not hand-written code.
- Designed the review workflow and rated 163 results by hand against four verdicts (accurate, partial, needs sources, hallucinated), plus review, moderation, quarantine, and conflict queues.
AI Collaboration
- Claude Code and Cursor stood in for an engineering team: 220 merged PRs, 145 database migrations, 249 automated tests.
- Architecture call: LangGraph over a raw prompt chain, so tool calls and iteration are explicit and inspectable.
- Trade-off discussion: model routing. Haiku for the cheap, high-volume plan and evaluate steps, Sonnet for the final answer.
- Capped refinement at 2 rounds to bound cost and latency.
- Put interaction logging in early (migration 4 of 145), so every later feature had something real to measure against.
Full breakdown
Problem & Risk
Value RiskFeasibility RiskCompliance Risk
- Value risk: generic web and YouTube search returns a wall of results with no judgment about a specific family, the kids' ages, or a limited time window. Without trust there is no product.
- Feasibility risk: most "AI-powered" travel tools are a single prompt in a nice UI. Unclear whether a solo, AI-built agent could stay grounded.
- Compliance risk: logging real questions with location context meant retention and privacy had to be designed in, not added later.
Customer Discovery & Validation
Evidence
- Lived it first-hand as a worldschooling parent: the "we found out afterward" moment is the recurring failure.
- Hand-curated the first 300 entries before automating anything, to learn what a trustworthy entry looks like.
- Earlier hallucination issues in the resource database led to a verified-source-first policy, direct evidence that trust, not content volume, is the binding constraint for this audience.
- Put the product in front of 20 real users and changed it based on what they said.
- Change 1: removed the AI-generated image from the front page and replaced it with a carousel of real pictures from providers.
- Change 2: removed images over privacy concerns and added a way to upload images with consent.
The information exists. Timely, trustworthy delivery doesn't. Users pushed the product the same way: real over generated, consent over convenience.
Context
- Rebuilt the platform on Next.js, Supabase, and Vercel, and imported the legacy catalog from the old site.
- The agent searches the catalog alongside the open web and YouTube, and labels where each source came from.
- Kept scope narrow: search and trust first. Parent accounts, newsletter, and community tips are not built yet.
Timeline
git log --oneline⏱ 10 weeks, 220 PRs
kickoff: first commit, rebuild on Next.js + SupabaseJul 27
foundation: 26 PRs in 5 days, legacy catalog importedJuly
build: 146 PRs: full taxonomy, public directory, entity profiles, promotion pipelineAugust
iterate: 48 more PRs merged, latest is #235Sep to Oct
feedback: 20 real users: AI image out, provider photo carousel in, image uploads with consentOngoing
next: fixed eval set from the 163 rated results, human review of the resource queueNext
What We Did
- Built an agent that plans its own search, calls real tools (catalog, web, YouTube), and evaluates its results before deciding to refine or finalize.
- Grounding guardrail: a source URL only reaches the answer if it was a real search hit. If the model's URL doesn't match, confidence is lowered.
- Labelled every source by origin (catalog, YouTube, blog, social) and added a confidence line to the answer card.
- Human in the loop on the catalog: agent discovery writes to a pending queue and needs a human approve. A weekly Haiku pre-screen scores unreviewed resources, and fixes are never applied automatically.
- Full tracing: every question stores its reasoning trace (step, tools, query, outcome, sufficient or not) and final results. 2,616 results logged across blog, video, web, social, and community sources.
- Human eval: 163 of those results rated by hand. 122 accurate, 36 partial or off-target, 3 needing sources, 2 hallucinated.
- Privacy by design: request-log IPs stored as HMAC hashes, 90-day retention with a daily purge, image uploads only with consent.
~ rough sketch, not an actual screenshot ~
plan
pick tools + queries
search
catalog, web, YouTube
evaluate
good enough?
finalize
grounded answer + sources
refine
max 2 rounds
trace logged
every step, every run
human rating
accurate / hallucinated / partial
Who Benefits
- Traveling parents get an answer with sources they can open, not a wall of generic search results.
- Creators, providers, and groups get a profile page they can claim.
- The platform gets a trust layer that scales: human review where it matters most, AI pre-screen for the long tail.
Next Steps
- Turn the 163 hand-rated results into a fixed eval set and re-score it on every prompt or model change.
- Bring down the 22% partial or off-target results, the biggest quality gap in the ratings.
- Work down the resource review queue and resolve the open AI pre-screen flags, to measure how often the AI reviewer agrees with a human.
- Check the answer summary against its source, not only the URL.
- Populate Events: the table and admin flow exist, the public tab is still empty.