VisualLM 2.0: real 3D, multi-provider AI, faster + self-validating generation#2
Open
Maxpeng59 wants to merge 18 commits into
Open
VisualLM 2.0: real 3D, multi-provider AI, faster + self-validating generation#2Maxpeng59 wants to merge 18 commits into
Maxpeng59 wants to merge 18 commits into
Conversation
…pairs
Reduce runtime crashes that exhaust the repair loop, and cut the latency of
both generation and repair, so fewer scenes hit the slow LLM repair path and
the ones that do converge (or fail fast) instead of grinding the whole budget.
Sandbox (sandbox-worker.js):
- Invented helpers (H/cam/view) degrade to a chainable no-op in any chain
order; cam/view rebind to the proxy so real().invented() is safe too.
- Per-frame error tolerance: a scene that renders then throws on an occasional
frame keeps running; only a first-frame failure or ~0.5s of sustained
throwing escalates to repair.
- Heartbeat on the first clean frame (load "ready" signal ~300ms sooner).
- offendingLine(): pull the exact failing source line out of the stack.
Client (app.js, index.html):
- MAX_REPAIRS 3->2; forward the offending line to /api/repair.
- Non-convergence guards: bail when a repair repeats the same error or returns
unchanged code, instead of grinding the full (slow) repair budget.
- Bump app.js cache-buster v10->v11 (was stranding all client changes behind
the stale ?v=10 for returning users).
Server (main.py):
- Generator-aware retry budget (cloud=2, local Ollama=3).
- Credit ".axes(" as a label so well-labeled plots stop triggering false
"unlabeled" regenerations.
- VISUALLM_CLAUDE_EFFORT / VISUALLM_CLAUDE_MAX_TOKENS env knobs (default high/32k).
- _repair_hint(): error-class fix guidance + a minimal-change directive, wired
into every repair provider; fold the offending line into the repair error.
Tests: +5 (repair hints, repair-visualization line folding). Suite 36->41, green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two production-robustness gaps in the public HTTP server, independent of the generation→repair work on this branch: - do_POST dispatched to handlers with no catch-all, so an UNEXPECTED exception (a bug, a new SDK error type — distinct from the RuntimeErrors that handlers already turn into 503s) escaped, dropped the client connection, and spilled a bare traceback to stderr with no response. Now logged and returned as a clean 500. - send_json's writes weren't guarded, so a client disconnecting mid-response (common during a slow generation) surfaced BrokenPipeError as an unhandled traceback. Now swallowed. Test: +1 (unexpected handler exception -> 500). Suite 41->42, green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The two complaints — "still errors after 3 tries" and "took a long time" — both trace to the same root: the browser was the validator, so every bad scene cost a full client round-trip + a slow model repair. This restructures the pipeline around catching and fixing errors WITHOUT model calls. - validate_scene.js: headless server-side scene validator. Runs untrusted generated code in a hardened Node `vm` (fresh empty context — no process/ require/import reachable; classic constructor.constructor escape contained; hard 2s timeout) and reports throws / blank / no-motion / no-labels in ~50ms. - main.py evaluate_scene(): single source of truth, validator-authoritative, static-gate fallback when node is absent. _try_generate now repairs fatal scenes SERVER-SIDE (exact runtime error folded into the prompt) so the browser receives an already-runnable scene. - autofix_code(): deterministically prefixes bare Math.* calls / PI / TAU outside strings & comments — fixes the #1 "X is not defined" throw with ZERO model round-trips. Wired into sanitize_code. - scene_library.py + library_match(): curated, hand-verified STEM corpus with keyword retrieval. Strong match short-circuits to an instant, known-correct scene (Fourier prompt: 5ms vs a 10-60s generation). This is the practical, retrieval-augmented realization of "feed it STEM data". - Exact-prompt cache of validated scenes (instant repeats), only caches clean scenes. plan_visualization chain: cache -> strong library -> models -> library fallback -> placeholder. - Frontend: elapsed-seconds ticker during generation; curated/cached badges. - Dockerfile installs Node so the validator runs in production too. - Health reports validator + library_size. 66 tests pass (24 new). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The weak local model's most common SEMANTIC failure isn't a crash — it's drawing real content in the wrong coordinate space (e.g. v.line(v.X(x), ...), which double-maps every point off the canvas). It runs, it "paints", it has labels, so the old checks passed it — but the screen shows only axes. The validator now tracks, per content draw, whether it lands on-canvas (data- space view/cam methods convert via X/Y or project() first, exactly like the real helpers). paint>0 with zero on-screen => onscreen:false, which evaluate_scene treats as fatal and repairs with a coordinate-space hint. 'paint' now counts CONTENT draws only (grid/axes/background are scaffold), so an axes-only "scene" is correctly seen as blank. All 6 library scenes pass; 72 tests green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- scene_library_generated.json: 42 scenes authored by the stem-scene-library multi-agent workflow (one author per domain + adversarial per-scene review), every one re-validated through validate_scene.js (runs, paints on-screen, animates, labeled). Covers ~24 domains: quantum, linear algebra, waves, algorithms, astronomy, neuroscience, control systems, probability, etc. - Hand-written electromagnetism scenes (dipole field lines, RC circuit) to cover the two UI example chips the workflow batch missed. - library_match: weight title/tag tokens, drop stopwords, dominance gap so an ambiguous near-tie doesn't fast-path; high-confidence matches fast-path even when a duplicate-topic scene also scores well. Library fallback now requires real relevance (>=2.0) instead of any score, so a failed generation never shows a wrong-topic curated scene. - Dockerfile copies the generated JSON; README documents the reliability/speed architecture + how to regenerate the library. 69 tests pass. 7/10 example chips now render instantly from the verified library; the rest generate. Verified in-browser: dipole + 3D moon-phases scenes render correctly. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Watched the local model write a Doppler scene with pos = t*4 (unbounded): the source and all wavefronts sail off the right edge within seconds, leaving only a static observer line. It passed validation because early frames (t<=3) still had content on-screen. Now the validator samples a late frame (t=30) and tracks the FINAL frame's on-screen fraction: if <15% of the last frame's content is on canvas, the scene "drifted off and never loops" -> fatal, repaired with a hint to bound/loop the motion (t % period, Math.sin(t)). All 50 library scenes still pass; 71 tests green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ge conflicts) PR #1 was squash-merged into main (948adc3); this branch kept its original commits (b58ae9e, b426c44) plus newer work - validator drift-detection, the 50-scene verified library, and the stem-viz-plugin - so re-merging main conflicted on main.py, tests/test_main.py, and validate_scene.js. For all three the branch is a content-superset of main (it contains PR #1's exact commits, which main carries as the squash, plus additive work on top), so the conflicts were resolved in favor of the branch. Verified: 71/71 tests pass and the merged validator still runs. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Live re-run exposed the subtler case: the local model wrote xSource = -speed*t again, and fixed labels (title, origin marker) kept the last-frame on-screen fraction just above 15%, so the moving subject sailed off-screen undetected. - Validator now also flags a COLLAPSE: content abundant on-screen early (>=5 draws in some early frame) but the late frame retains <40% of that peak => the subject drifted away while only fixed annotations remain. Catches the exact generated Doppler; no false-flags across the 50-scene library. - System prompt Rule 1 now bans `x = speed*t` outright and requires looping/ bounded motion (Math.sin(t), t % period) so moving subjects stay in frame — fixing the root cause for every generation, not just repairs. - 72 tests pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Bake the product's teaching philosophy into the generator. Scenes must build
understanding of the mechanism (sweep the parameter, explain in labels, provoke
prediction) rather than just computing and displaying the answer to a student's
specific problem. Applied consistently to SCENE_SYSTEM_PROMPT ("Style and
pedagogy") and the stem-viz skill (SKILL.md + concepts.md).
Guarded against over-correction: scenes stay concrete and labeled, and a live
readout of a CHANGING quantity remains good teaching (it makes the relationship
visible). The line is illuminate-the-mechanism vs. hand-over-the-answer.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Make it runnable without touching the terminal after a one-time key setup. - launch.py: loads .env, picks a free port, starts main.py, waits until /api/health is green, then opens the UI in a chrome-less app window (Chrome/Edge/Brave --app, else the default browser). Flags: --no-browser, --smoke. - VisualLM.command: macOS double-click wrapper (chmod +x). - desktop.py: optional TRUE native window via pywebview (one pip install). - .env.example: API-key template (.env stays gitignored). - README: new "Run it (as an app)" section + Files entries. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
shutil.which("node") is `str | None`, and pyright/Pylance can't carry the
non-None proof across the separate node_validator_available() helper (it's a
module global), so the subprocess.run arg list was typed `list[str | None]`
(reportCallIssue + reportArgumentType). Rebind to a local and guard it inline
so it narrows to `str`. Behavior-identical — the inline check is De
Morgan-equivalent to the old `not node_validator_available()`. pyright now
reports 0 errors on main.py.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
make_app.sh assembles VisualLM.app — a proper bundle (Info.plist + a generated multi-resolution .icns from make_icon.py, a dependency-free stdlib PNG writer). Its launcher resolves the repo by the bundle's own location and opens VisualLM in a native window (pywebview if installed) or an app-style browser window otherwise. The python3 path is baked at build time so a Finder launch — which gets a minimal PATH — still finds the right interpreter. The .app is a git-ignored build artifact; run ./make_app.sh to (re)build. A fully self-contained PyInstaller bundle is the next step (in progress). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
app_main.py runs the VisualLM server IN-PROCESS (server in a daemon thread, window on the main thread) instead of spawning `python main.py` — required for a PyInstaller bundle, where sys.executable is the app binary, not a Python interpreter. main.py's BASE_DIR now resolves to sys._MEIPASS when frozen so the static assets + validate_scene.js load from the bundle (no-op unfrozen). Opens a native pywebview window if installed, else an app-style browser window. Verified: `app_main.py --smoke` serves /api/health; 72/72 tests green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a committed PyInstaller spec + build_app.sh that freeze app_main.py into a standalone double-click VisualLM.app — bundles Python, the static assets, validate_scene.js, and the scene library; runs the server in-process and serves from the bundle (BASE_DIR -> sys._MEIPASS). Verified: the frozen binary's `--smoke` boots the server and serves index.html from the bundle (~22 MB app). Also: app_main.py --smoke now confirms static assets serve (the real bundle-path test), and the packaged app reads .env from next to VisualLM.app or ~/.visuallm/ (when frozen, its __file__ is inside the bundle). Build artifacts (build/, dist/) stay git-ignored. 72/72 tests green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add tests/test_launcher.py (stdlib, headless): .env loading (quote-stripping, skipping comments/junk lines, never overriding an existing shell env var, missing-file no-op), free_port (returns the preferred port when free; falls back to a bindable port when it's taken), wait_healthy (true against a live 200 server, false on a dead port within the timeout), and app_main._env_files (unfrozen -> local .env; frozen -> alongside VisualLM.app + ~/.visuallm). The new launcher/app code was previously untested. Suite 72 -> 81, green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…n names
autofix_code blindly prefixed every bare PI / TAU / Math-fn name. A scene that
aliases one — `const PI = Math.PI` or `const TAU = 6.28` — was rewritten to
`const Math.PI = ...` (a SYNTAX error), and a local `function log(){}` /
`const log = ...` became `Math.log`. Now we collect the names the scene declares
itself (const/let/var/function) and never rewrite those; genuinely bare calls
and constants are still fixed. Regression test added; suite 81 -> 82, green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
_rewrite_declared_names renamed a reserved binding (H/ctx/t) after ANY comma in
a const/let/var span — including expression commas inside ()/[]/{}. So a very
common pattern like `const y = H.lerp(a, b, t);` became `H.lerp(a, b, _t)` (an
undefined reference), and `const p = [Math.cos(t), t]` lost its `t` — silently
feeding the server-side repair loop and burning round-trips. The docstring
already said "top level of the statement" but the regex never enforced bracket
depth.
Replaced the naive regex with a depth-aware scan that renames only genuine
declarators (at depth 0, right after const/let/var or a top-level comma).
Real redeclarations still rename (`const W = H.W, H = H.H` -> `_H`; `let H, ctx`
-> `let _H, _ctx`). Also removed a SyntaxWarning (a `\s` in a non-raw docstring).
2 regression tests; suite 82 -> 84, green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The `function scene(...) { ... }` unwrap (rfind-based) and the fence strip run
on every generated scene but were only tested on the trivial case. Verified and
pinned the tricky edges: nested arrow-function braces, a '}' inside a string
literal (rfind must still land on the function's own closing brace), and a
fenced + wrapped scene where both layers are stripped. No code change — coverage
only. Suite 84 -> 86, green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Makes VisualLM an industrial-grade app: real 3D rendering, multi-provider AI, and a generation pipeline that is fast and reliable instead of slow and error-prone. Production smoke-tested (binds 0.0.0.0 on injected PORT;
/api/healthreportsvalidator: true,library_size: 50).Real 3D (was flat point clouds)
surface3d/mesh3d, shadedcam.sphere, 3D lines/polys/grid/axes with numeric ticks, drag-to-orbit + scroll zoom.Right animation, faster + fewer errors
The root cause of "errors after 3 tries" + "slow" was that the browser was the validator, so each bad scene cost a full round-trip + a slow model repair. Restructured to catch/fix errors without model calls:
validate_scene.js— headless server-side validator in a hardened Nodevm(fresh empty context — noprocess/require/import; constructor escape contained; 2s timeout). Reports throws / blank / off-screen (data-vs-pixel coordinate mixups) / no-motion / no-labels in ~50ms. Repairs happen server-side; the browser receives a runnable scene. Degrades gracefully without Node.autofix_code()— deterministically prefixes bareMath.*/PI/TAUoutside strings; fixes the Refine generation→repair pipeline: fewer crashes, faster loads, converging repairs #1 "X is not defined" crash with zero model calls.scene_library.py+scene_library_generated.json, ~50 verified scenes across ~24 domains, authored by a multi-agent workflow + re-validated) with keyword retrieval: common prompts render in ~5ms instead of 10–60s. The practical, retrieval-augmented realization of "feed it STEM data."Multi-provider + production
PORT/0.0.0.0cloud support, Dockerfile (bundles Node) + Render blueprint.Verified in-browser
2D plots with tick values/legends/live readouts; shaded 3D surfaces, DNA helix, torus; dipole field lines; 3D moon-phases. Console clean.
🤖 Generated with Claude Code