feat(files): add OCR fallback for scanned PDFs

[AoLiGei1221]

Summary

Support scanned/image-only PDFs by adding an OCR fallback to the backend PDF text indexing pipeline. Today, PDFs that render visually but have no embedded text layer are uploaded and viewable, but AI cannot read them because file_pages.text stays empty and the user sees PDF context unavailable.

Product Rationale

This matters for the core education/exam workflow:

• Admin task instruction PDFs may be scans of exams, worksheets, or school handouts.
• User document source PDFs may come from scanned/phone-captured material.
• Users expect AI to understand a PDF that is visibly readable in the viewer.
• Since Document/Task PDFs are locked after creation, uploading a scanned PDF without AI-readable context is hard for the user to recover from.

Acceptance Criteria

• When normal PDF.js text extraction returns zero usable text pages, the backend attempts OCR before marking the file unavailable.
• OCR text is persisted into the existing text index path so AI retrieval can use it through file_pages and existing downstream retrieval/chunking behavior.
• The AI panel readiness indicator can distinguish at least:
  • PDF context processing
  • PDF context available
  • PDF context unavailable
• Preferably, UI/API can also expose whether context came from OCR, for example available via OCR, without overstating OCR accuracy.
• OCR failures are logged with enough operational detail to debug, while user-facing text remains concise.
• Existing pure-text PDF behavior remains unchanged.
• Existing Task multi-PDF behavior remains unchanged.
• Existing single-source-PDF-per-Document behavior remains unchanged.

Suggested Implementation Notes

• Keep the current PDF.js extraction as the first path.
• Add OCR fallback only when extracted text is empty or below a conservative threshold.
• Render PDF pages to images server-side for OCR.
• Start with a bounded MVP to avoid runaway cost/latency:
  • page count cap or background job limit
  • file size/page image size safeguards
  • clear timeout and retry behavior
• Store OCR-derived text in the same retrieval-compatible index path, but consider recording provenance so the UI/AI prompt can treat OCR text as less authoritative than embedded text.

Decisions Needed Before Implementation

• OCR engine/provider:
  • local open-source OCR such as Tesseract
  • cloud OCR such as Google Vision / Document AI
  • hybrid local-first/cloud-later approach
• Whether OCR runs synchronously during upload indexing or asynchronously in a background job.
• Whether to add DB metadata for text source/provenance, for example embedded text vs OCR.
• Cost and page limits for enterprise usage.

Test Plan

• Add a fixture scanned/image-only PDF whose PDF.js text extraction returns zero items.
• Verify indexing falls back to OCR and writes non-empty file_pages.text.
• Verify AI readiness becomes available after OCR succeeds.
• Verify pure text PDFs still use the existing extraction path.
• Verify OCR failure leaves the file in a clear unavailable/failed state without breaking upload/viewing.

Out Of Scope

• Perfect handwriting recognition.
• Full layout reconstruction.
• Manual PDF replacement after Document/Task creation.
• Changing the existing one-PDF Document rule or Task multi-PDF support.

[AoLiGei1221]

Need to be discussed for this issue. It will be hold here for now.

[ShenzheZhu]

Current main audit confirms this issue is still valid and not implemented. FileTextIndexService.buildFileTextIndexPublicationPlan() marks a PDF unavailable when all extracted pages are empty, and the repository has no OCR engine/provider or OCR fallback path.

This remains separate from #911: visual PDF delivery, range loading, and embedded-text indexing can be healthy while scanned/image-only PDFs still have no AI-readable context. Keep this open pending the provider/execution/provenance decisions already listed in the issue body.