This tutorial demonstrates how to build a production-grade insurance card extraction pipeline using Pydantic AI and LangGraph for high-precision data validation and structured output.

Introduction

The manual verification of insurance cards is a significant bottleneck in healthcare administration, often leading to data entry errors, delayed claims processing, and increased operational costs. Traditional Optical Character Recognition (OCR) solutions, which rely on template-based matching or simple text extraction (e.g., Tesseract), frequently fail when confronted with the high variability of insurance card layouts, overlapping text, or low-quality scans.

To address these challenges, modern Intelligent Document Processing (IDP) leverages Vision Language Models (VLMs) and agentic workflows. By combining Pydantic AI for structured data extraction and LangGraph for stateful orchestration, developers can build systems that not only extract text but also validate it against business rules and external databases in real-time.

This tutorial provides a technical deep dive into building an automated insurance card verification system. We will move beyond simple extraction to implement a multi-stage pipeline that includes image preprocessing, structured extraction, automated validation, and human-in-the-loop (HITL) triggers for low-confidence results.

Objectives

By the end of this tutorial, you will:

  1. Architect a multi-stage IDP pipeline using Python and LangGraph.
  2. Define complex data schemas for insurance information using Pydantic.
  3. Implement a Vision-based extraction agent using Pydantic AI.
  4. Design a stateful validation logic to handle low-confidence extractions.
  5. Configure a human-in-the-loop workflow for manual verification triggers.

Prerequisites

To follow this tutorial, you will need the following:

  • Python 3.12+: The latest stable version of Python is recommended for optimal Pydantic and typing support. Python Downloads
  • Pydantic AI: For defining structured extraction agents. Pydantic AI Documentation
  • LangGraph: For orchestrating the agentic workflow. LangGraph Documentation
  • OpenAI API Key: Specifically for access to gpt-4o or gpt-4o-mini (Vision-capable models).
  • Docker: For containerizing the application and managing dependencies. Docker Documentation

Implementation

1. Project Architecture and State Definition

In an agentic IDP system, the "State" is the source of truth that moves through the graph. For insurance verification, the state must track the raw image, the extracted data, validation flags, and a confidence score.

We use LangGraph's StateGraph because it allows for cyclical transitions—for example, if a validation step fails, the graph can route the task back to the extraction node with specific instructions on what to correct.

2. Defining the Data Schema

The first step is defining what we expect to extract. Insurance cards typically contain a member name, ID number, group number, plan type, and provider information. We use Pydantic models to enforce type safety and provide the LLM with a clear structure.

from typing import Optional, List
from pydantic import BaseModel, Field, validator

class ProviderInfo(BaseModel):
    name: str = Field(description="The name of the insurance provider, e.g., Aetna, BlueCross.")
    phone: Optional[str] = Field(description="Customer service or provider phone number.")
    website: Optional[str] = Field(description="Provider website URL.")

class InsuranceCardModel(BaseModel):
    member_name: str = Field(description="Full name of the primary member.")
    member_id: str = Field(description="Unique identification number for the member.")
    group_number: Optional[str] = Field(description="Group identification number.")
    plan_type: Optional[str] = Field(description="Type of plan, e.g., PPO, HMO, EPO.")
    effective_date: Optional[str] = Field(description="The date the coverage became active.")
    issuer_id: Optional[str] = Field(description="The 80840 issuer identification number.")
    is_valid_format: bool = Field(default=True, description="Internal flag for format validation.")

    @validator("member_id")
    def validate_id_length(cls, v):
        if len(v) < 5:
            raise ValueError("Member ID is too short to be valid.")
        return v

3. Implementing the Extraction Agent

We utilize Pydantic AI's Agent class to handle the interaction with the Vision LLM. Unlike standard LangChain calls, Pydantic AI focuses on ensuring the output strictly adheres to the defined Pydantic model.

import os
from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIModel

# Initialize the model
model = OpenAIModel('gpt-4o', api_key=os.getenv("OPENAI_API_KEY"))

# Define the extraction agent
extraction_agent = Agent(
    model,
    result_type=InsuranceCardModel,
    system_prompt=(
        "You are a specialized medical billing assistant. "
        "Extract all relevant information from the provided insurance card image. "
        "If a field is not legible, do not guess; leave it as null. "
        "Ensure the member_id is captured exactly as written, including any prefixes."
    ),
)

async def extract_card_data(image_url: str) -> InsuranceCardModel:
    """
    Sends the image to the Vision LLM and returns a structured Pydantic model.
    """
    result = await extraction_agent.run(
        f"Please process this insurance card: {image_url}",
    )
    return result.data

4. Orchestrating the Workflow with LangGraph

The power of this system lies in the orchestration. We define a graph where the nodes represent discrete steps: extraction, validation, and human review.

The following code block demonstrates the construction of the StateGraph. We include a validation node that checks the extracted member_id against a hypothetical database or a set of regex patterns.

from typing import TypedDict, Annotated
from langgraph.graph import StateGraph, END

class AgentState(TypedDict):
    image_path: str
    extracted_data: Optional[InsuranceCardModel]
    validation_errors: List[str]
    requires_human_review: bool
    confidence_score: float

def extraction_node(state: AgentState):
    """Node to handle the initial OCR and extraction."""
    # In a real scenario, you would pass the image bytes or URL
    data = extraction_agent.run_sync(f"Process: {state['image_path']}").data
    
    # Simple confidence heuristic based on missing fields
    missing_fields = [k for k, v in data.model_dump().items() if v is None]
    confidence = 1.0 - (len(missing_fields) / 10.0)
    
    return {
        "extracted_data": data,
        "confidence_score": confidence,
        "requires_human_review": confidence < 0.8
    }

def validation_node(state: AgentState):
    """Node to validate extracted data against business logic."""
    data = state["extracted_data"]
    errors = []
    
    if data and not data.member_id.isalnum():
        errors.append("Member ID contains invalid characters.")
    
    # Logic to flag for human review if validation fails
    review_needed = state["requires_human_review"] or len(errors) > 0
    
    return {
        "validation_errors": errors,
        "requires_human_review": review_needed
    }

def human_review_node(state: AgentState):
    """Placeholder for human-in-the-loop intervention."""
    # In production, this would trigger a webhook or update a DB for a UI to pick up
    print(f"CRITICAL: Manual review required for {state['image_path']}")
    return state

def route_after_validation(state: AgentState):
    """Conditional logic to determine the next step."""
    if state["requires_human_review"]:
        return "human_review"
    return END

# Build the graph
workflow = StateGraph(AgentState)

workflow.add_node("extract", extraction_node)
workflow.add_node("validate", validation_node)
workflow.add_node("human_review", human_review_node)

workflow.set_entry_point("extract")
workflow.add_edge("extract", "validate")
workflow.add_conditional_edges(
    "validate",
    route_after_validation,
    {
        "human_review": "human_review",
        END: END
    }
)
workflow.add_edge("human_review", END)

app = workflow.compile()

5. Architectural Decisions and Rationale

Why Pydantic AI over LangChain?

While LangChain is a versatile framework, Pydantic AI is purpose-built for structured data. It leverages Python's type hints more natively, reducing the boilerplate required to ensure that an LLM's response matches a specific schema. In healthcare, where data integrity is paramount, this strictness is a feature, not a limitation.

Why LangGraph for Orchestration?

Traditional linear pipelines (A -> B -> C) fail when data is ambiguous. LangGraph allows for "cycles." For example, if the validation_node identifies that the member_id is missing a required prefix common to a specific provider, the graph can loop back to the extraction_node with a refined prompt: "The provider is Aetna; please re-examine the image for a member ID starting with 'W'."

Vision LLMs vs. Traditional OCR

The following table compares the performance characteristics of different extraction methodologies:

| Metric | Traditional OCR (Tesseract) | Vision LLM (GPT-4o) | Agentic IDP (Proposed) | | : | : | : | : | | Layout Flexibility | Low (Requires Templates) | High (Generalization) | High (Context-Aware) | | Accuracy (Handwriting) | Poor | Good | Excellent (with loops) | | Processing Speed | Very Fast (<1s) | Moderate (2-5s) | Slow (5-10s) | | Structured Output | Manual Parsing Required | Native JSON | Validated Pydantic Models | | Cost per Document | Near Zero | $0.01 - $0.05 | $0.02 - $0.07 | | Reliability | Low (Fragile) | Medium (Hallucinations) | High (Self-Correcting) |

6. Handling GDPR and PII

In a European enterprise context, handling insurance cards involves processing Protected Health Information (PHI) and Personally Identifiable Information (PII). When implementing this system:

  1. Data Minimization: Only send the necessary image fragments to the LLM if possible.
  2. Encryption: Ensure all images are encrypted at rest and in transit.
  3. Private Deployments: For high-compliance environments, utilize Azure OpenAI or Google Vertex AI with regional data residency (e.g., francecentral or germanywestcentral) to ensure data does not leave the EEA.
  4. Audit Logs: Use LangGraph's state persistence to maintain an audit trail of how data was extracted and who (if anyone) performed the manual override.

7. Deployment with Docker

To ensure consistency across environments, the application should be containerized.

# Dockerfile
FROM python:3.12-slim

WORKDIR /app

COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY . .

CMD ["python", "main.py"]

The requirements.txt should include:

  • pydantic-ai
  • langgraph
  • openai
  • python-dotenv
  • pillow (for image preprocessing)

Conclusion

Automating insurance card verification requires more than just text extraction; it requires a system capable of reasoning about the data it perceives. By combining the structured enforcement of Pydantic AI with the sophisticated orchestration of LangGraph, developers can build IDP pipelines that significantly reduce manual overhead while maintaining high levels of accuracy. This agentic approach allows for self-correction and seamless human intervention, making it suitable for the rigorous demands of the healthcare industry.

Azura AI helps enterprises architect and scale these intelligent document processing systems, ensuring high-precision extraction and full compliance with regional data regulations.