HealthRAG

Personal Medical Document System (Built for a Friend)

30 minutes → 3 seconds • Medical records you can actually use

A friend spent 10+ years visiting doctors without a diagnosis. Each specialist saw only their fragment — missing connections between symptoms across clinics and years. Built HealthRAG in 12 days. Processed 870 documents. Now she answers any doctor's question about her history in 3 seconds instead of searching through papers for 30 minutes.

Before/After

Before: Doctor asks "What was your hemoglobin six months ago?" → 30 minutes searching papers, trying to remember which clinic

After: Ask Telegram bot → 3 seconds, complete answer with dates and trend

Impact: 870 documents from 10+ years, instantly searchable. Trends visible that were impossible to see before (e.g., "vitamin D dropping for 2 years").

How It Works

Step 1: Send photo of medical document to Telegram bot

Step 2: AI reads it (including handwriting), extracts test results, dates, diagnoses. When it encounters unknown abbreviations ("What does 'Hb' mean?"), asks you once, then remembers forever.

Step 3: Data organized into two systems: structured database for trends ("show cholesterol last 2 years") + search engine for context ("find all mentions of thyroid")

Result: Ask any question about your medical history, get answer in 3 seconds with specific numbers and dates.

Value & Scale

Solved for: 1 person with chronic health condition (10+ years without diagnosis)

Potential market: 133M Americans with chronic conditions visiting multiple specialists

Time saved: 30 minutes → 3 seconds per doctor's question. At 12 appointments/year with 10 questions each = 60 hours saved annually

Processing cost: $19 to digitize 10 years of medical history (870 documents)

What Makes It Special

System learns medical terminology from you, not from generic databases — it adapts to how your specific doctors write and abbreviate.

Technical Architecture

Data Pipeline:

Medical Document → Google Document AI (OCR) → Claude AI (Structuring + Normalization) → JSON Schema Validation → Parallel Write to BigQuery (structured data) + Qdrant (embeddings) → Conversational AI Layer

Key Technical Decisions:

  • Google Document AI: Superior OCR for medical documents with complex layouts (processing all 870 documents cost ~$19)
  • Claude 3.5 Sonnet: Best-in-class for medical entity extraction and normalization
  • BigQuery: Powerful analytics engine for health trends, cost-effective at scale (10GB free tier)
  • Qdrant: Self-hosted vector database for privacy-sensitive medical data
  • Hybrid Storage: Structured data in BigQuery (fast queries), full context in Qdrant (semantic search)

Schema Consistency Challenge:

Different labs use different formats and abbreviations. Solution: AI normalizes to unified schema, mapping variations ("Hb", "HGB", "Hemoglobin") to standard LOINC codes. System learns from user corrections to improve accuracy over time.

Data Quality & Validation:

  • Cross-reference extracted values against medical knowledge base
  • Flag impossible values (e.g., negative blood pressure)
  • Handle edge cases: partial data, unclear handwriting, non-standard formats
  • Transaction management: atomic writes prevent partial data corruption

Real Numbers

Performance:

  • 95% OCR accuracy, 90% extraction accuracy
  • 15-30 seconds per document (OCR + structuring + storage)
  • ~$0.05-0.15 per document processed
  • < 2 seconds to answer questions
  • 870 documents processed, 10+ years of history

What Actually Changed:

  • Finding specific test results: 30 minutes → 3 seconds
  • Can now see health trends that were invisible before (e.g., "vitamin D has been dropping for 2 years")
  • Doctor visits: my friend can instantly answer any question about past results
  • Helps doctors make better decisions with complete historical context

Key Challenge

Medical terminology chaos: 50+ labs write "hemoglobin" 50 different ways ("Hb", "HGB", "Hemoglobin", different units). Built system that asks user once when it sees unknown term, then standardizes everything automatically. This was harder than the AI part.

Tech Stack

Python, FastAPI, Telegram Bot API, Claude 3.5 Sonnet, Claude 4 Opus, Claude Code, Google Document AI, BigQuery, Qdrant, OpenAI Embeddings, PostgreSQL

~21,000 lines of code across 35 modules. Built in 12 days, refined over actual use.