Skip to content

XY-881: [ELF benchmark P1] Pin Docker cold-start embedding dependency blockers#163

Merged
yvette-carlisle merged 3 commits into
mainfrom
y/elf-xy-881
Jun 10, 2026
Merged

XY-881: [ELF benchmark P1] Pin Docker cold-start embedding dependency blockers#163
yvette-carlisle merged 3 commits into
mainfrom
y/elf-xy-881

Conversation

@yvette-carlisle

Copy link
Copy Markdown
Member

Summary

  • Pin the OpenViking Docker local embedding dependency path to llama-cpp-python==0.3.28 from the CPU wheel index with binary-only install before .[local-embed].
  • Preserve install/import/setup failures as incomplete, while classifying reached add_resource/find evidence misses as wrong_result/retrieval_wrong_result.
  • Refresh production-ops fixtures, external adapter manifest, report docs, README claims, and benchmark tests for the new boundary.

Verification

  • jq empty apps/elf-eval/fixtures/real_world_memory/production_ops/cold_start_missing_dependency_incomplete.json apps/elf-eval/fixtures/real_world_external_adapters/memory_projects_manifest.json
  • bash -n scripts/live-baseline-benchmark.sh
  • docker compose -f docker-compose.baseline.yml config
  • ELF_BASELINE_PROJECTS=OpenViking cargo make baseline-live-docker
  • cargo make real-world-memory-production-ops
  • cargo make real-world-memory
  • cargo test -p elf-eval --test real_world_job_benchmark
  • cargo make fmt
  • cargo make lint-fix
  • cargo make checks

…mbedding benchmark boundary","authority":"XY-881"}
…with benchmark vNext main","authority":"XY-881"}
…t with live sweep wording","authority":"XY-881"}
@yvette-carlisle yvette-carlisle merged commit a580246 into main Jun 10, 2026
10 checks passed
@yvette-carlisle yvette-carlisle deleted the y/elf-xy-881 branch June 10, 2026 09:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant