PDF Accessibility App

An automated PDF remediation tool from the CUNY AI Lab that converts uploaded PDFs into accessible, PDF/UA-1 compliant documents.

Upload a PDF and the app classifies it, runs OCR if needed, extracts structure, generates alt text, writes PDF/UA tags, and validates the result against veraPDF. Documents that can't be fully remediated automatically are flagged for manual review.

Features

• Automatic classification of digital, mixed, and scanned PDFs
• OCR with language auto-detection via OCRmyPDF and Tesseract
• Structure extraction via Docling, with optional remote docling-serve for GPU acceleration
• LLM-generated alt text for figures, with decorative-image detection
• LLM-assisted semantic tagging for tables, forms, reading order, and grounded text
• Deterministic PDF/UA tag writing via pikepdf
• Compliance gating with veraPDF PDF/UA-1 validation
• Fidelity checks for text drift, reading order, table coverage, and form labels
• Anonymous sessions — no login required, jobs scoped to an HTTP-only cookie

Tech Stack

Layer	Technology
Backend	Python 3.12, FastAPI, SQLAlchemy (async SQLite)
Frontend	React, TypeScript, Vite, Tailwind CSS 4, TanStack Query
PDF Processing	pikepdf, OCRmyPDF, Ghostscript, Poppler, QPDF
Structure Extraction	Docling (local or docling-serve)
Semantic Analysis	Open-weight vision models through the CAIL Gateway
Validation	veraPDF

Requirements

• Python 3.12+ and uv
• Bun
• Ghostscript
• OCRmyPDF
• Tesseract
• Poppler (pdftoppm)
• veraPDF and a Java runtime
• A CAIL Gateway API key

On macOS, install system dependencies via Homebrew. On Debian/Ubuntu, install ghostscript, poppler-utils, tesseract-ocr, and a Java runtime.

Quick Start

# Clone and configure
git clone <repo-url> pdf-accessibility-app
cd pdf-accessibility-app
cp .env.example .env
# Edit .env and set LLM_API_KEY (a CAIL Gateway key)

# Install dependencies
cd backend && uv sync
cd ../frontend && bun install

Run the two services in separate terminals:

# Backend
cd backend
uv run uvicorn app.main:app --reload --port 8001

# Frontend
cd frontend
bun dev

Open http://localhost:5173. The frontend proxies /api and /health to the backend on port 8001.

Docker

A single-container image bundles all system dependencies and serves the built frontend from FastAPI.

cp .env.example .env
# Set LLM_API_KEY

docker compose up -d --build

Open http://localhost:8080. Set APP_PORT in .env to use a different port.

To run without Compose:

docker build -t pdf-accessibility-app .
docker run -d \
  --name pdf-accessibility-app \
  --env-file .env \
  -p 8080:8001 \
  -v pdf_accessibility_data:/app/data \
  -v pdf_accessibility_cache:/home/app/.cache \
  pdf-accessibility-app

By default, the Docker image is slim and expects DOCLING_SERVE_URL to point to a remote docling-serve instance. Set WITH_LOCAL_DOCLING=true before building to include local Docling and preload models into the image. For subpath deployments (e.g., behind a reverse proxy), set VITE_APP_BASE_PATH=/pdf-accessibility/ before building.

Tesseract language packs included: English, Spanish, French, German, Chinese (Simplified and Traditional), Russian, Arabic, Korean, Bengali, Polish, Hebrew, Yiddish, Haitian Creole, Hindi, Italian, Portuguese, and Japanese. Add others by extending the Dockerfile.

Version 2 (Worker behind Doorway)

Version 2 serves the app at https://tools.ailab.gc.cuny.edu/pdf-accessibility/ for signed-in CUNY people. A Cloudflare Worker (worker/) owns the app, its API, jobs, and files; the pipeline runs on actual-dell (python -m app.runner) and takes work from the Worker. Each person's model calls use their own Gateway budget. See docs/architecture.md, deploy/actual-dell/README.md, and worker/README.md.

NML Deployment

The NML host is reached with the local nml-ssh wrapper. The deployed project lives at /data/pdf-accessibility-app and runs as the Docker Compose project pdf-accessibility-app.

The NML override uses host networking because Docker bridge networking is not reliable on that VM. The app listens on 127.0.0.1:8001/0.0.0.0:8001, and Nginx serves it publicly at https://tools.ailab.gc.cuny.edu/pdf-accessibility/.

Required NML .env settings:

ANONYMOUS_SESSION_COOKIE_SECURE=true
CORS_ALLOW_ORIGINS=https://tools.ailab.gc.cuny.edu
VITE_APP_BASE_PATH=/pdf-accessibility/
LLM_BASE_URL=https://tools.ailab.gc.cuny.edu/v1
LLM_MODEL=qwen3-vl-235b-a22b-instruct
LLM_API_KEY=<the app's sk-cail Gateway key>
DOCLING_SERVE_URL=<actual-dell's Tailscale Serve URL, port 8444>
WITH_LOCAL_DOCLING=false

The app no longer reads GEMINI_API_KEY or USE_DIRECT_GEMINI_PDF; delete them from the server's .env. Without a real LLM_API_KEY the app refuses to start.

Deploy from the NML host:

cd /data/pdf-accessibility-app
git pull
docker compose up -d --build
curl -fsS http://127.0.0.1:8001/health
curl -fsS http://127.0.0.1:8001/health/ready
curl -fsS https://tools.ailab.gc.cuny.edu/pdf-accessibility/health/ready

Do not commit .env or the backup .env.* files on the server.

Configuration

Configure the app via .env. Key variables:

Variable	Description	Default
LLM_BASE_URL	Chat-completions base URL (the CAIL Gateway)	https://tools.ailab.gc.cuny.edu/v1
LLM_API_KEY	CAIL Gateway key (required)	—
LLM_MODEL	Open-weight vision model on the Gateway	qwen3-vl-235b-a22b-instruct
ALT_TEXT_MAX_CONCURRENCY	Max concurrent alt-text requests per PDF	8
ALT_TEXT_GLOBAL_MAX_CONCURRENCY	Process-wide cap for alt-text requests	12
MAX_UPLOAD_SIZE_BYTES	Maximum raw upload size accepted by the app	104857600
MAX_UPLOAD_PAGES	Maximum pages accepted for self-service processing	300
MAX_UPLOAD_PAGE_RENDER_PIXELS	Maximum estimated pixels for one page rendered at UPLOAD_PREFLIGHT_RENDER_DPI	75000000
MAX_UPLOAD_IMAGE_PIXELS	Maximum embedded image pixels on one page	75000000
MAX_UPLOAD_TOTAL_IMAGE_PIXELS	Maximum embedded image pixels across the PDF	1000000000
MAX_UPLOAD_IMAGE_HEAVY_PAGES	Maximum pages with at least UPLOAD_IMAGE_HEAVY_PAGE_MIN_PIXELS image pixels	75
MAX_UPLOAD_OCR_SCAN_TOTAL_IMAGE_PIXELS	Stricter embedded scan-image pixel cap for OCR/Paper Capture PDFs	150000000
MAX_UPLOAD_OCR_SCAN_IMAGE_HEAVY_PAGES	Stricter image-heavy page cap for OCR/Paper Capture PDFs	25
MAX_FILES_PER_UPLOAD	Maximum PDFs accepted in one upload request	5
MAX_ACTIVE_JOBS_PER_SESSION	Maximum queued/processing jobs per anonymous browser session	3
MAX_ACTIVE_JOBS_GLOBAL	Maximum queued/processing jobs accepted across the app	12
MAX_CONCURRENT_JOBS	Maximum PDF pipeline jobs actively executing in-process	2
OCRMYPDF_JOBS	OCRmyPDF worker count per PDF; keep low on shared hosts	1
OCRMYPDF_MAX_IMAGE_MPIXELS	OCRmyPDF image decompression ceiling per image	75
WITH_LOCAL_DOCLING	Include local Docling in Docker builds	false
DOCLING_SERVE_URL	Remote docling-serve URL (falls back to local Docling when unset)	—
DOCLING_SERVE_TOKEN	Bearer token for a protected docling-serve proxy	—
OCR_LANGUAGE	Fallback Tesseract language code	eng
JOB_TTL_HOURS	Hours before jobs expire	12
CSRF_PROTECTION_ENABLED	Require CSRF header for cookie-authenticated write requests	true
VERAPDF_PATH	Path to the veraPDF binary	verapdf
GHOSTSCRIPT_PATH	Path to the Ghostscript binary	gs

Structure extraction with docling-serve

Structure extraction is the slowest pipeline step. Running a persistent docling-serve process eliminates cold starts and enables GPU acceleration. Start it with:

DOCLING_DEVICE=mps docling-serve run --host 0.0.0.0 --port 5001

Then set DOCLING_SERVE_URL=http://localhost:5001 in .env. Without DOCLING_SERVE_URL, the app falls back to local Docling on CPU.

OCR language detection

The app auto-detects document language during classification and selects the matching Tesseract pack for OCR. Digital and mixed PDFs use lingua-py on the extracted text. Scanned PDFs probe OCR on page 1 with every installed language pack, then run lingua-py on the result. If no pack is installed for the detected language, the app falls back to OCR_LANGUAGE.

On macOS, brew install tesseract-lang installs all packs. On Debian/Ubuntu, install individual packs with apt install tesseract-ocr-<lang>.

Pipeline

Each upload runs through seven steps:

1. Classify — Determine whether the PDF is digital, mixed, or scanned
2. OCR — Add a searchable text layer to scanned pages
3. Structure — Extract document structure via Docling, with LLM-assisted TOC enhancement
4. Alt Text — Generate alt text for figures and reclassify misidentified elements
5. Tag — Resolve ambiguous semantics (tables, forms, reading order) via LLM, then write PDF/UA tags deterministically
6. Validate — Check PDF/UA-1 compliance with veraPDF
7. Fidelity — Verify output faithfulness (text drift, reading order, table coverage, form labels)

Project Structure

backend/
  app/
    api/              FastAPI route handlers
    pipeline/         Classify, OCR, structure, tag, validate, fidelity
    services/         Semantic adjudication, storage, LLM client
    models.py         SQLAlchemy ORM models
    config.py         App settings
  tests/

frontend/
  src/
    pages/            Upload, Dashboard, JobDetail, Review
    components/
    api/              TanStack Query hooks
    types/

data/                 Runtime storage (git-ignored)

Development

# Backend tests
cd backend
PYTHONPATH=. uv run pytest tests -q

# Frontend lint and build
cd frontend
bun run lint
bun run build

From the repository root, run the credential-free backend/frontend check:

./scripts/check.sh

Verify the effective runtime (LLM provider, Docling target, installed binaries) with:

cd backend
PYTHONPATH=. uv run python scripts/runtime_diagnostics.py

When the app is running, /health is a lightweight liveness check and /health/ready verifies runtime dependencies needed for real PDF processing: database connectivity, writable storage, required PDF binaries, LLM configuration, and Docling/docling-serve availability. A remote docling-serve is probed at most once per 30 seconds per app process; readiness requests inside that window reuse the last result, so repeated calls to the public endpoint do not reach docling-serve.

Documentation

• Architecture
• Benchmarks
• Accessibility Coverage
• Accessibility Coverage Matrix
• PDF/UA Rule Coverage
• Backend README
• Frontend README

SSO Note (tools.ailab.gc.cuny.edu deployment)

The CUNYLogin SSO gateway being rolled out on tools.ailab.gc.cuny.edu currently classifies /pdf-accessibility/ as public — the anonymous-session, zero-retention workflow is intentionally preserved. If that policy changes, the proxy hands this app a short-lived signed X-CAIL-Identity-JWT assertion, which the backend verifies itself; see backend/CAIL_IDENTITY.md for the contract and activation inputs. Bare X-CAIL-* headers are never trusted as identity. The anonymous cookie remains separate (CSRF and anonymous-session isolation), while verified job ownership keys off the assertion's stable subject.

Models: every AI step calls the CAIL Gateway at LLM_BASE_URL with rendered page images, never PDF files. The Gateway admits only open-weight models and meters spend per key. Per-person Gateway credentials come with the Doorway version of the app; see docs/cail-integration-plan.md.