Skip to content

docs: launch-readiness pass for the local inference feature#241

Open
quiet-node wants to merge 22 commits into
mainfrom
docs/local-inference-launch
Open

docs: launch-readiness pass for the local inference feature#241
quiet-node wants to merge 22 commits into
mainfrom
docs/local-inference-launch

Conversation

@quiet-node

Copy link
Copy Markdown
Owner

Overview

Aligns every user-facing word in the repo with the shipped local inference feature for the v0.15.0 release. The bundled built-in engine is the zero-setup default, with Ollama as the optional provider. This pass corrects stale copy, fills the documentation gap, and refreshes the in-app tips.

What changed

In-app copy

  • The model reasoning capability is named "Reasoning" consistently: the capability badge, the live streaming block ("Reasoning..." then "Reasoning"), the /think description, and the chat export.
  • Residency wording is unified on "Keep Warm" and "in memory" (Apple Silicon shares unified memory, so there is no separate VRAM).
  • The OCR-failure hint no longer says "via Ollama", the no-models cue drops "LLM", and onboarding no longer calls a model a "brain".

README

  • Leads with the built-in zero-setup story and names the Settings > Models surfaces: Library, Discover (Staff picks and Browse all), and Providers, plus Keep Warm.
  • Corrects the model-download pointer to Discover and the TypeScript badge.

Docs

  • New guides: Models and providers, Privacy, and Troubleshooting, linked from the README.
  • configurations.md, SECURITY.md, and the tuning and OCR guides are corrected for accuracy and made engine-agnostic. configurations.md drops the phantom MAX_IMAGES_PER_MESSAGE constant and documents the real, tunable image cap.
  • SECURITY.md documents the local inference posture: a localhost-only sidecar with the web UI disabled, bounded GGUF parsing, and model provenance from pinned repo revisions.

Tips

  • Stale Ollama tips are generalized to the built-in engine, with new tips for the model library, providers, vision, and reasoning.

Provider scope

  • The OpenAI-compatible provider (its UI is behind a dev flag) is intentionally not mentioned in user-facing copy for this release. The loader still accepts an existing kind = "openai" config for back-compat.

Testing

bun run test:all:coverage (frontend 100%, 2075 tests; backend 100% lines) and bun run validate-build both pass.

Notes

The README's demo videos and logo show older UI and need a human recapture (visuals cannot be verified headless). The CHANGELOG's hand-written Unreleased block carries the BREAKING migration notes, so only its factual contradiction was corrected here; how release-please folds it into the 0.15.0 section is a release-engineering choice left to the release.

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
… LLM label

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
… wording

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
…-Ollama shared limits

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
…asoning

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
…e search guide

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
…drop migration history

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
…enter

Signed-off-by: Logan Nguyen <lg.131.dev@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant