How to Get Reliable Structured Output via Ollama

Why Structured Output Is Hard With Local Models

You need JSON from your local model. Not prose with JSON somewhere in it. Not almost-valid JSON. Actual, parseable, schema-conformant JSON that you can feed directly into your application logic.

This is harder than it sounds. Local models vary wildly in their instruction-following capabilities. Some models reliably output JSON when asked. Others wrap it in markdown code blocks. Others hallucinate extra keys, drop required fields, or produce syntactically invalid output that breaks on the first bracket mismatch.

The failure modes are predictable:

• Invalid JSON syntax — trailing commas, unquoted keys, single quotes instead of double quotes
• Hallucinated keys — the model invents fields you did not ask for
• Missing required fields — the model decides some fields are optional when they are not
• Type mismatches — you asked for an integer, you got a string containing a number
• Nested prose — the model embeds the JSON inside an explanation or wraps it in markdown

Production pipelines cannot tolerate this variance. You need a pattern that gives you reliable, validated output regardless of which model you are running.

The Schema-First Approach

The solution is to define your output schema explicitly and use it at both generation time and validation time.

Pydantic gives you schema definitions that serve double duty: they describe what you want to the model, and they validate what the model returns. If the model output does not match the schema, Pydantic raises a validation error that you can catch and handle.

from pydantic import BaseModel

class ArticleMetadata(BaseModel):
    title: str
    summary: str
    tags: list[str]
    reading_time_minutes: int

This schema is your contract. The model must produce output that matches it. If the model fails, you retry. If retries fail, you surface the error rather than passing garbage downstream.

Ollama's Native JSON Mode

Ollama supports a format="json" option that constrains the model to output valid JSON. This is not the same as schema validation—it ensures syntactic validity, not semantic correctness—but it eliminates the most common failure mode.

import ollama

response = ollama.chat(
    model="llama3.2",
    messages=[{
        "role": "user",
        "content": "Extract the title and author from this text. Return JSON with keys 'title' and 'author'.\n\nText: The Great Gatsby by F. Scott Fitzgerald is a classic."
    }],
    format="json"
)

print(response['message']['content'])
# {"title": "The Great Gatsby", "author": "F. Scott Fitzgerald"}

The format="json" parameter tells Ollama to constrain sampling to valid JSON tokens. The model cannot output prose or malformed syntax. It will produce valid JSON or fail entirely.

This is necessary but not sufficient. The JSON will be valid, but it might not match your schema.

Adding Validation With Pydantic

The production pattern combines Ollama's JSON mode with Pydantic validation and retry logic. If the model returns JSON that does not match your schema, retry with a fresh generation.

import ollama
from pydantic import BaseModel, ValidationError
import json

class ArticleMetadata(BaseModel):
    title: str
    summary: str
    tags: list[str]
    reading_time_minutes: int

def extract_metadata(text: str, max_retries: int = 3) -> ArticleMetadata:
    schema = ArticleMetadata.model_json_schema()

    for attempt in range(max_retries):
        response = ollama.chat(
            model="llama3.2",
            messages=[{
                "role": "user",
                "content": f"""Extract metadata from this text.
Return ONLY valid JSON matching this schema:
{json.dumps(schema, indent=2)}

Text: {text[:2000]}"""
            }],
            format="json"
        )

        try:
            data = json.loads(response['message']['content'])
            return ArticleMetadata(**data)
        except (json.JSONDecodeError, ValidationError) as e:
            if attempt == max_retries - 1:
                raise RuntimeError(f"Failed after {max_retries} attempts: {e}")

    raise RuntimeError("Max retries exceeded")

This pattern handles both JSON syntax errors and schema validation errors. The model_json_schema() method generates a JSON Schema from your Pydantic model, which you include in the prompt to show the model exactly what structure you expect.

Prompt Engineering for Consistency

The prompt matters. Models are more likely to follow your schema if you show them an example first.

def extract_with_example(text: str, max_retries: int = 3) -> ArticleMetadata:
    example = ArticleMetadata(
        title="Example Article",
        summary="A brief description of the article content.",
        tags=["python", "tutorial"],
        reading_time_minutes=5
    )

    for attempt in range(max_retries):
        response = ollama.chat(
            model="llama3.2",
            messages=[{
                "role": "user",
                "content": f"""Extract metadata from text. Return JSON only.

Example output:
{example.model_dump_json(indent=2)}

Now extract from this text:
{text[:2000]}"""
            }],
            format="json"
        )

        try:
            data = json.loads(response['message']['content'])
            return ArticleMetadata(**data)
        except (json.JSONDecodeError, ValidationError):
            if attempt == max_retries - 1:
                raise

    raise RuntimeError("Max retries exceeded")

The few-shot example gives the model a concrete target to match. This is more effective than abstract schema descriptions for many models.

Handling Partial Failures Gracefully

Sometimes the model gets close but not quite right. You might want to extract what you can rather than failing entirely.

from pydantic import BaseModel, field_validator
from typing import Optional

class ArticleMetadataLenient(BaseModel):
    title: str
    summary: str
    tags: list[str] = []  # Default to empty if missing
    reading_time_minutes: Optional[int] = None

    @field_validator('reading_time_minutes', mode='before')
    @classmethod
    def coerce_reading_time(cls, v):
        if v is None:
            return None
        if isinstance(v, str):
            # Handle "5 minutes" or "5"
            import re
            match = re.search(r'\d+', v)
            return int(match.group()) if match else None
        return int(v)

This lenient schema uses defaults and validators to handle common model mistakes. The field_validator coerces string-encoded numbers to integers. Missing optional fields get default values instead of raising errors.

Use lenient schemas for enrichment tasks where partial data is better than no data. Use strict schemas for tasks where incomplete output is worse than an error.

Model Selection for Structured Output

Not all models are equally good at structured output. As a general rule:

• Smaller models (3B-7B) work well for simple schemas with few fields and no nesting. They are fast and cheap to run.
• Medium models (8B-14B) handle moderate complexity—nested objects, arrays of objects, conditional fields.
• Larger models (30B+) are necessary for complex schemas with deep nesting, many fields, or subtle constraints.

Test on your actual schema before committing to a model. A 7B model might handle 90% of your schemas perfectly and fail on the remaining 10%.

MODELS_BY_COMPLEXITY = {
    "simple": "llama3.2",      # 3B, fast, handles flat schemas
    "moderate": "llama3.1:8b", # 8B, good balance
    "complex": "qwen2.5:32b",  # 32B, handles nested structures
}

def select_model(schema: type[BaseModel]) -> str:
    fields = schema.model_fields
    has_nested = any(
        hasattr(f.annotation, 'model_fields')
        for f in fields.values()
    )
    field_count = len(fields)

    if has_nested or field_count > 8:
        return MODELS_BY_COMPLEXITY["complex"]
    elif field_count > 4:
        return MODELS_BY_COMPLEXITY["moderate"]
    return MODELS_BY_COMPLEXITY["simple"]

When to Use This Pattern

This pattern is appropriate when:

• Article enrichment — extracting metadata, tags, summaries from saved articles
• Content classification — categorizing documents into predefined types
• Entity extraction — pulling structured data from unstructured text
• Data transformation — converting prose descriptions into structured records
• Pipeline preprocessing — preparing text for downstream structured processing

It is not appropriate when:

• You need the model's reasoning visible (structured output hides the thought process)
• The schema is too complex for reliable generation (consider breaking into smaller schemas)
• Latency is critical and retries are unacceptable (use hosted APIs with guaranteed structured output)

Complete Example: Article Processor

Here is a complete, production-ready example that processes articles and extracts structured metadata:

import ollama
from pydantic import BaseModel, field_validator
from typing import Optional
import json
import hashlib

class ProcessedArticle(BaseModel):
    title: str
    summary: str
    main_topics: list[str]
    key_takeaways: list[str]
    technical_level: str  # "beginner", "intermediate", "advanced"
    estimated_reading_minutes: int

    @field_validator('technical_level')
    @classmethod
    def validate_level(cls, v):
        allowed = {"beginner", "intermediate", "advanced"}
        v_lower = v.lower().strip()
        if v_lower not in allowed:
            return "intermediate"  # Default fallback
        return v_lower

class ArticleProcessor:
    def __init__(self, model: str = "llama3.2"):
        self.model = model
        self.schema = ProcessedArticle.model_json_schema()

    def process(self, text: str, max_retries: int = 3) -> ProcessedArticle:
        prompt = f"""Analyze this article and extract structured metadata.
Return ONLY valid JSON matching this schema:
{json.dumps(self.schema, indent=2)}

Article text:
{text[:4000]}"""

        for attempt in range(max_retries):
            response = ollama.chat(
                model=self.model,
                messages=[{"role": "user", "content": prompt}],
                format="json"
            )

            try:
                data = json.loads(response['message']['content'])
                return ProcessedArticle(**data)
            except (json.JSONDecodeError, Exception) as e:
                if attempt == max_retries - 1:
                    raise RuntimeError(f"Processing failed: {e}")

        raise RuntimeError("Max retries exceeded")

# Usage
processor = ArticleProcessor()
article = processor.process("""
Your article text here...
""")
print(article.model_dump_json(indent=2))

This pattern gives you reliable, schema-validated structured output from any local model. The combination of Ollama's JSON mode, Pydantic validation, and retry logic handles the variance inherent in local model output.