Medical Anamnesis Chatbot with NLP (Chatbot PUSTU)

Project Overview

Chatbot PUSTU is a full-stack medical anamnesis chatbot for Indonesian Puskesmas (community health centers). Built for a Natural Language Processing course, it uses Multinomial Naive Bayes with TF-IDF vectorization to reach 92.61% intent classification accuracy across 14 Indonesian medical intent classes — covering the full pipeline from LLM-powered data generation to a custom NER system, stateful dialog management, and 24/7 cloud deployment.

Healthcare Problem

Indonesian Puskesmas struggle with slow, inconsistent patient anamnesis: 15+ minutes of manual history-taking per patient, non-standardized records, and a lack of robust Indonesian medical NLP tooling — all while small clinics cannot afford commercial EMR systems.

Technical Architecture

Three-tier system with an ML-powered backend:

• Frontend (Vercel) — Next.js 16 + TypeScript chat UI with conversation history, dark/light mode, and client-side PDF export.
• Backend (Railway) — Flask + Gunicorn serving intent classification, entity extraction, dialog state, and smart prefilling.
• ML pipeline — Multinomial Naive Bayes (alpha=0.1) over TF-IDF features (5,000 features, 1–2 n-grams), custom preprocessing, and dictionary-based NER (97 symptoms, 23 body locations) with hybrid keyword boosting.

Training Data Generation (Gemini Flash 2.0)

With no existing Indonesian medical anamnesis dataset, I generated 14,000 balanced samples (1,000 each across 14 intents) using Gemini Flash 2.0 with prompt engineering tuned for colloquial Indonesian medical language — total cost roughly $3.50 on the free tier.

The 14 intents span the full interview: chief complaint, accompanying symptoms, duration, location, severity, medical history, medications, allergies, and risk factors, plus conversational intents (greeting, thanks, confirmation, denial, unclear).

NLP Preprocessing (Built From Scratch)

Because Indonesian NLP libraries lack medical vocabulary and slang handling, I built a custom pipeline: lowercasing, punctuation removal, tokenization, slang normalization (94 mappings, e.g. gak → tidak, udah → sudah), and stopword filtering (93 words).

Example: "Dok saya batuk gak sembuh-sembuh udah 3 hari" becomes "batuk tidak sembuh sudah 3 hari".

Custom Named Entity Recognition

With no pre-trained Indonesian medical NER available, entities are extracted via dictionary and regex matching: a 97-type symptom dictionary (with synonyms), 23 body locations, and 3 severity levels. Duration extraction uses context keywords (sudah, sejak, selama, sekitar) to avoid false positives — "batuk sudah 3 hari" is captured while "umur saya 28 tahun" is correctly rejected.

NER validation (500-sample manual test):

Intent Classification

Naive Bayes was chosen for sub-10ms inference, roughly 2s training on 14K samples, native handling of sparse TF-IDF features, and interpretable probabilistic outputs for confidence thresholding.

Test set (2,800 samples): 92.61% accuracy with 93% macro precision, recall, and F1. Strongest intents reach 98% F1 (greeting, medication history, unclear); weakest are chief complaint (84%, confused with accompanying symptoms) and severity (89%, ambiguous keywords).

A hybrid layer boosts ML predictions with keyword evidence (0.90–0.95 confidence thresholds), lifting effective accuracy to roughly 94%.

Dialog Management

A 14-stage stateful interview runs from greeting and demographics through complaint, symptoms, duration, location, severity, history, medications, allergies, and risk factors to a final summary. A smart prefilling algorithm auto-fills later stages when a user volunteers information early — "sakit kepala parah sudah 3 hari di bagian kanan" fills symptom, severity, duration, and location at once, cutting average interview time by roughly 30%.

Deployment

The backend runs on Railway (Python 3.10, Gunicorn, models loaded in memory); the frontend on Vercel (global CDN, Next.js 16 / React 19). Both use automatic HTTPS with 24/7 availability.

Key Takeaways

• LLM-generated data is a cost-effective substitute for manual annotation in low-resource domains (roughly $3.50 vs $1,000+).
• Domain-specific dictionaries beat generic Indonesian NLP libraries for medical NER (90%+ F1, no training).
• Classical ML (Naive Bayes + TF-IDF) still delivers production accuracy with roughly 100x faster inference than BERT and zero GPU cost.
• Hybrid prediction and confidence thresholding reduce error rate and enable graceful "ask for clarification" fallbacks.

Live Demo: pustu-anamnesis-chatbot.vercel.app · Source: GitHub Repository · Course: Natural Language Processing, Hasanuddin University, 2025