diff --git a/docs/superpowers/plans/2026-10-22-python-bindings-move.md b/docs/superpowers/plans/2026-10-22-python-bindings-move.md deleted file mode 100644 index eb8c9469..00000000 --- a/docs/superpowers/plans/2026-10-22-python-bindings-move.md +++ /dev/null @@ -1,396 +0,0 @@ -# Python Bindings Move Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** Move `bindings/python` from `ArcadeData/arcadedb`, with its history, into this repo. Make it build and pass CI here against `arcadedata/arcadedb:latest`, then delete it upstream. - -**Architecture:** -- `git filter-repo` extracts `bindings/python/**` and the two Python workflows from a fresh upstream clone and moves the bindings to the repo root. -- The result is merged into a branch here, and the monorepo coupling is removed. -- The ArcadeDB version now comes from the `arcadedb-engine-.jar` name in the jars directory, not from `pom.xml`. The Docker tag defaults to `latest`. - -**Tech Stack:** git-filter-repo, bash, Docker (buildx), Python 3.10+ (setuptools, pytest, JPype), GitHub Actions. - -**Spec:** `docs/superpowers/specs/2026-10-22-python-bindings-move-design.md` - -## Global Constraints - -- **Source:** `ArcadeData/arcadedb` `main` at extraction time. After extraction, record its SHA in the spec header. `bindings/python` is frozen upstream from then on. -- **History:** keep all of it. Rewrite issue refs in commit messages from `#NNNN` to `ArcadeData/arcadedb#NNNN`. -- **Layout:** bindings at the repo root; workflows in `.github/workflows/`. -- **Docker tag:** default `latest`, overridable by the `ARCADEDB_TAG` environment variable. No `pom.xml` anywhere and no fallback to one. -- **Versions:** the Maven→PEP 440 mapping is unchanged: `X.Y.Z-SNAPSHOT` → `X.Y.Z.devN` (N from the existing tracker), `X.Y.Z-RCn` → `X.Y.Zrcn`, a release stays as it is, and `--python-patch=N` adds `.postN`. -- **Out of scope:** - - Behaviour changes to bindings, tests or examples (path changes only). - - Edits to the `pyproject.toml` humemai URLs and authors. - - PyPI publishing. - - Upstream's `e2e-python/`. -- **Merging:** the PR here must be merged with a **merge commit**, never squash or rebase, or the history is lost. -- **Upstream:** nothing is deleted upstream until both workflows are green on `main` here. Opening the upstream PR needs the user's go-ahead. -- **Commit trailer:** `Co-Authored-By: Claude Opus 5.5 `. - -## Review Focus - -- **Concurrent snapshot publish:** `latest` is republished while a CI run is in flight. All jobs in one run must build against the same ArcadeDB version. The `download-jars` job resolves the version once and hands it to every later job as `ARCADEDB_TAG` (Task 5). -- **Bad jars directory:** a directory with no `arcadedb-engine-*.jar`, or with engine jars of two versions, must fail the build with a clear error, not produce a `0.0.0` or wrong-version wheel (Task 3 tests). -- **Pinned tag:** `ARCADEDB_TAG` set to a release such as `26.10.2` gives wheel version `26.10.2`. An RC tag gives `rc` (Task 3 tests). -- **Local jars:** `build.sh linux/amd64 3.12 /path/to/jars` with the default `latest` tag checks the integration jar against the version read from those jars, not against the string `latest` (Task 4 verification). -- **Native builds:** the macOS/Windows path in `build-native.sh` takes its version from the downloaded jars directory, because there is no `pom.xml` to read (Task 4, exercised by the CI matrix in Task 7). - ---- - -## File Structure (after the move) - -``` -.github/workflows/test-python-bindings.yml # moved; paths/triggers rewritten (Task 5) -.github/workflows/test-python-examples.yml # moved; paths/triggers rewritten (Task 5) -.github/actionlint.yaml # new: the two upstream entries (Task 5) -.github/dependabot.yml # new: uv at "/" + github-actions (Task 5) -.pre-commit-config.yaml # new: black, isort, yaml, shfmt hooks (Task 5) -REUSE.toml # new: bindings annotation, root paths (Task 5) -.gitignore # merged root + bindings ignore rules (Task 1) -README.md # bindings README + provenance/version note (Task 6) -pyproject.toml, setup.py # moved -scripts/extract_version.py # reads version from jars dir (Task 3) -scripts/write_version.py # takes jars dir (Task 3) -scripts/build.sh, build-native.sh, Dockerfile.build # root build context, latest tag (Tasks 2, 4) -src/arcadedb_embedded/, src/java/ # moved -tests/ # moved; + test_extract_version.py (Task 3) -tests/fixtures/ # 5 files copied from upstream integration/src/test/resources (Task 2) -examples/ # moved; 07–12 paths fixed (Task 2) -docs/superpowers/{specs,plans}/ # this spec and plan -``` - ---- - -### Task 1: Extract history and merge it into a branch - -**Files:** -- Create: everything under the File Structure above that comes from upstream (via merge) -- Modify: `.gitignore`, `README.md` (merge conflict resolution), `docs/superpowers/specs/2026-10-22-python-bindings-move-design.md` (SHA) - -**Interfaces:** -- Produces: branch `move-python-bindings` with upstream history at the root, and `EXTRACTED_SHA`, the upstream `main` commit it came from. Task 8 uses this SHA. - -- [ ] **Step 1: Create the branch and commit the spec and plan** - -```bash -git checkout -b move-python-bindings -git add docs/superpowers && git commit -m "docs: design and plan for moving the Python bindings here" -``` - -- [ ] **Step 2: Install filter-repo and make a fresh single-branch clone in the scratchpad** - -Run: `brew install git-filter-repo && git clone --single-branch --branch main https://github.com/ArcadeData/arcadedb.git "$SCRATCH/arcadedb-extract"`. Then record `EXTRACTED_SHA=$(git -C "$SCRATCH/arcadedb-extract" rev-parse HEAD)`. - -- [ ] **Step 3: Filter the clone** - -```bash -cd "$SCRATCH/arcadedb-extract" -git filter-repo \ - --path bindings/python/ \ - --path .github/workflows/test-python-bindings.yml \ - --path .github/workflows/test-python-examples.yml \ - --path-rename bindings/python/: \ - --message-callback 'import re; return re.sub(rb"(? -git remote add extract "$SCRATCH/arcadedb-extract" && git fetch extract -git merge --allow-unrelated-histories extract/main -m "Import Python bindings history from ArcadeData/arcadedb@$EXTRACTED_SHA" -``` - -Resolve the add/add conflicts: -- **`README.md`:** take the bindings version. Task 6 adds the note. -- **`.gitignore`:** take the union, with the bindings block placed **last** so its `!src/**/` re-include wins. Remove the dead line `bindings/python/src/arcadedb_embedded/_version.py`. - -Then `git remote remove extract`. - -- [ ] **Step 5: Verify history and ignore rules** - -Run each of these: -- `git log --oneline --follow src/arcadedb_embedded/core.py | wc -l`. Expected: more than 1, with upstream commits listed. -- `git status --ignored --short src/ scripts/ tests/`. Expected: no tracked-source path reported as ignored. -- `git check-ignore -v local-jars/lib/x.jar`. Expected: ignored, which is intended. - -- [ ] **Step 6: Record the SHA in the spec and commit** - -Replace the `Source:` SHA in the spec header with `$EXTRACTED_SHA`, then `git commit -am "docs: record extracted upstream SHA"`. - ---- - -### Task 2: Rewrite monorepo paths to the root layout, and add test fixtures - -**Files:** -- Modify: `scripts/Dockerfile.build`, `scripts/build.sh`, `scripts/build-native.sh`, `scripts/setup_jars.py`, `tests/test_import_database.py:66-75`, `examples/02,07,08,09,10,11,12_*.py`, comments in `src/arcadedb_embedded/results.py:210`, `src/arcadedb_embedded/type_conversion.py:77`, `tests/test_java_package_shadowing.py`, `tests/test_vector_params_verification.py` -- Create: `tests/fixtures/{importer-vertices.csv,importer-edges.csv,neo4j-export-mini.jsonl,importer-word2vec.txt,importer-rdf.xml}`, copied byte for byte from upstream `integration/src/test/resources/` at `EXTRACTED_SHA` - -**Interfaces:** -- Consumes: Task 1 branch. -- Produces: the Docker build context is the repo root, and `Dockerfile.build` copies `src/`, `tests/`, `scripts/...`, `local-jars/lib/`, `pyproject.toml`, `setup.py` and `README.md` from there. Task 4 relies on this. - -- [ ] **Step 1: Copy the five fixtures and point `_resource_path(name)` at `Path(__file__).resolve().parent / "fixtures" / name`** - -- [ ] **Step 2: Rewrite paths** - -Make these changes: -- **`Dockerfile.build`:** drop `bindings/python/` from every `COPY` source and comment. -- **`build.sh`:** change the two `../..` build contexts to `"$PY_BINDINGS_DIR"`. Change the header example to `./scripts/build.sh linux/amd64 3.12 /path/to/arcadedb/package/target/arcadedb-.dir/arcadedb-/lib`. -- **Examples 07–12:** - - `parents[3]` → `parents[1]` - - `repo_root / "bindings/python/dist"` → `repo_root / "dist"` - - `"/workspace/bindings/python/..."` → `"/workspace/..."` - - the `"...in bindings/python/dist..."` messages → `"...in dist..."` -- **Example 02's docstring:** `cd examples`. -- **`setup_jars.py:144`:** drop the `cd bindings/python` hint. -- **The four comments:** `bindings/python/src/java` → `src/java`, and `bindings/python/src` → `src`. - -Leave the `"bindings/python/docs is not present on this branch"` test-data string in `tests/test_check_test_skips.py:53` as it is. - -- [ ] **Step 3: Verify that no monorepo paths remain** - -Run: `git grep -n -E 'bindings/python|parents\[3\]|\.\./\.\.' -- . ':!docs/superpowers' ':!.github/workflows'` -Expected: only `tests/test_check_test_skips.py:53`. The workflows are Task 5. `README.md` is Task 6 and may still match. - -- [ ] **Step 4: Commit** — `refactor: build from the repo root instead of the monorepo; vendor importer fixtures` - ---- - -### Task 3: Read the ArcadeDB version from the jars - -**Files:** -- Modify: `scripts/extract_version.py`, `scripts/write_version.py` -- Test: `tests/test_extract_version.py` (new; loads the script with `importlib`, the same way `tests/test_check_test_skips.py` does) - -**Interfaces:** -- Produces, in `scripts/extract_version.py`: - - `extract_raw_version_from_jars(jars_dir: Path) -> str`. Returns the `` from `arcadedb-engine-.jar`. Raises `ValueError` when there is no engine jar, or when there are engine jars of more than one version. - - `maven_to_pep440(raw: str, patch_version: int = 0, increment_dev: bool = False) -> str`. This is the existing conversion body of `extract_version_from_pom`, moved here unchanged. - - `extract_version(jars_dir: Path, fmt: str = "pep440", patch_version: int = 0, increment_dev: bool = False) -> str`. With `fmt="docker"` it returns the raw version. - - CLI: `python3 scripts/extract_version.py [JARS_DIR] [--format=pep440|docker] [--python-patch=N] [--increment-dev]`. The default `JARS_DIR` is `/src/arcadedb_embedded/jars`. - - `extract_raw_version_from_pom` and `extract_version_from_pom` are deleted, and the module docstring is updated to say "engine jar name" instead of "pom.xml". -- Produces, in `scripts/write_version.py`: CLI `python3 scripts/write_version.py [JARS_DIR]` (same default). It writes `src/arcadedb_embedded/_version.py`, and the generated docstring says "from the bundled ArcadeDB engine jar". - -- [ ] **Step 1: Write the failing tests** - -```python -# helper: _jars(tmp_path, *names) creates empty files with those names and returns tmp_path -def test_snapshot_engine_jar_gives_raw_version(tmp_path): - d = _jars(tmp_path, "arcadedb-engine-26.11.1-SNAPSHOT.jar", "arcadedb-server-26.11.1-SNAPSHOT.jar", "jline-3.30.0.jar") - assert mod.extract_raw_version_from_jars(d) == "26.11.1-SNAPSHOT" - assert mod.extract_version(d, fmt="docker") == "26.11.1-SNAPSHOT" - -def test_snapshot_converts_to_dev(tmp_path, monkeypatch): - # point the tracker at tmp_path so the repo's tracker is untouched - monkeypatch.setattr(mod, "get_dev_version_tracker_path", lambda: tmp_path / "t.json") - assert mod.extract_version(_jars(tmp_path, "arcadedb-engine-26.11.1-SNAPSHOT.jar")) == "26.11.1.dev0" - -def test_release_and_rc_and_post(tmp_path): - assert mod.maven_to_pep440("26.10.2") == "26.10.2" - assert mod.maven_to_pep440("26.10.2", patch_version=1) == "26.10.2.post1" - assert mod.maven_to_pep440("26.11.1-RC1") == "26.11.1rc1" - -def test_no_engine_jar_raises(tmp_path): - with pytest.raises(ValueError, match="arcadedb-engine"): - mod.extract_raw_version_from_jars(_jars(tmp_path, "arcadedb-server-26.11.1-SNAPSHOT.jar")) - -def test_two_engine_versions_raise(tmp_path): - d = _jars(tmp_path, "arcadedb-engine-26.11.1-SNAPSHOT.jar", "arcadedb-engine-26.10.2.jar") - with pytest.raises(ValueError, match="26.10.2.*26.11.1-SNAPSHOT|26.11.1-SNAPSHOT.*26.10.2"): - mod.extract_raw_version_from_jars(d) - -def test_cli_prints_docker_format(tmp_path): - d = _jars(tmp_path, "arcadedb-engine-26.11.1-SNAPSHOT.jar") - out = subprocess.run([sys.executable, str(SCRIPT_PATH), str(d), "--format=docker"], capture_output=True, text=True, check=True) - assert out.stdout.strip() == "26.11.1-SNAPSHOT" -``` - -- [ ] **Step 2: Run the tests and confirm they fail** - -Run: `python3 -m pytest tests/test_extract_version.py -v -p no:cacheprovider --noconftest`. Expected: FAIL with `AttributeError ... extract_raw_version_from_jars`. If `tests/conftest.py` needs the installed wheel, `--noconftest` keeps these pure-Python tests runnable without it. - -- [ ] **Step 3: Implement the interfaces above.** Match engine jars with `^arcadedb-engine-(.+)\.jar$`. A CLI `ValueError` or `FileNotFoundError` prints `Error: ...` to stderr and exits with code 1, as it does today. - -- [ ] **Step 4: Run the tests and confirm they pass** (same command). Expected: 6 passed. - -- [ ] **Step 5: Commit** — `feat: derive the ArcadeDB version from the bundled engine jar instead of pom.xml` - ---- - -### Task 4: Wire `latest` into the build scripts - -**Files:** -- Modify: `scripts/build.sh:151-154, 404`, `scripts/build-native.sh:311, 330`, `scripts/Dockerfile.build:193-194, 210, 231` - -**Interfaces:** -- Consumes: Task 3 CLIs. The Task 2 build context. -- Produces: `build.sh` honours `ARCADEDB_TAG` (default `latest`). Task 5's workflows set it. - -- [ ] **Step 1: Make the changes** - -| File | Change | -|---|---| -| `build.sh:151-154` | `DOCKER_TAG="${ARCADEDB_TAG:-latest}"`. Delete the "Detecting version from pom.xml" lines and print `Docker tag: $DOCKER_TAG`. | -| `build.sh:404` | Pass `ARCADEDB_VERSION="$(python3 "$SCRIPT_DIR/extract_version.py" "$LOCAL_JARS_DIR" --format=docker)"` instead of `$DOCKER_TAG`. | -| `build-native.sh:311` | `extract_version.py "$JARS_DIR" --format=pep440` | -| `build-native.sh:330` | `write_version.py "$JARS_DIR"` | -| `Dockerfile.build` | Delete `COPY pom.xml` and its comment. Change to `write_version.py /build/jars` and `extract_version.py --format=pep440 /build/jars`. The log text becomes "from engine jar". | - -- [ ] **Step 2: Verify the build against `latest`, with no arcadedb checkout involved** - -Run: `docker pull arcadedata/arcadedb:latest && ./scripts/build.sh linux/amd64 3.12` - -Expected: -- The build succeeds and the Docker tester stage passes. -- `ls dist/` shows `arcadedb_embedded-.dev0-*.whl`. -- `-SNAPSHOT` equals the output of `docker run --rm --entrypoint ls arcadedata/arcadedb:latest /home/arcadedb/lib | grep arcadedb-engine`. - -- [ ] **Step 3: Verify a pinned tag and the local-jars path** - -Run each of these: -- `ARCADEDB_TAG= ./scripts/build.sh linux/amd64 3.12`. Expected: the same wheel version. -- Copy the jars out of the image into `$SCRATCH/jars`, then run `./scripts/build.sh linux/amd64 3.12 "$SCRATCH/jars"`. Expected: the "Verifying embedded local integration JAR" step passes. - -- [ ] **Step 4: Run the full test suite against the built wheel on the host** - -Run: `python3 -m venv $SCRATCH/venv && $SCRATCH/venv/bin/pip install dist/*.whl ".[test]"` (if the extras resolve from the source tree, otherwise list them), then `$SCRATCH/venv/bin/pytest tests -q`. Expected: all pass. That includes `tests/test_import_database.py`, which reads from `tests/fixtures`, and `tests/test_extract_version.py`. - -- [ ] **Step 5: Commit** — `build: default to arcadedata/arcadedb:latest, overridable via ARCADEDB_TAG` - ---- - -### Task 5: Workflows and repository config - -**Files:** -- Modify: `.github/workflows/test-python-bindings.yml`, `.github/workflows/test-python-examples.yml` -- Create: `.github/actionlint.yaml`, `.github/dependabot.yml`, `.pre-commit-config.yaml`, `REUSE.toml` - -**Interfaces:** -- Consumes: `build.sh` reads `ARCADEDB_TAG` (Task 4). `extract_version.py JARS_DIR --format=docker` (Task 3). -- Produces: the `download-jars` job output `arcadedb-tag`, a concrete `X.Y.Z-SNAPSHOT`, which every later job passes to `build.sh` as `env: ARCADEDB_TAG`. - -- [ ] **Step 1: Change both workflows** - -**Triggers** -- `push` and `pull_request` with no `paths` filter. -- Keep `workflow_dispatch` and `workflow_call`. Keep the `build-version` input, but delete the `engine-from-source` inputs. -- Delete `workflow_run: Release`. -- Add `schedule: - cron: "0 3 * * *"` to `test-python-bindings.yml` and `"0 4 * * *"` to `test-python-examples.yml`. - -**Paths** -- Delete every `working-directory: bindings/python` and every `cd bindings/python`. -- Rewrite `bindings/python/` to ``. This covers artifact paths and the examples glob description. - -**`download-jars`** -- Delete the engine-from-source branch (`./mvnw ...`, `actions/setup-java`). -- `docker create arcadedata/arcadedb:latest`, copy the lib dir out, then `ARCADEDB_TAG=$(python3 scripts/extract_version.py src/arcadedb_embedded/jars --format=docker)`. -- Write `arcadedb-tag=$ARCADEDB_TAG` to `$GITHUB_OUTPUT` and declare it under `outputs:`. - -**`test` and `test-examples`** -- Delete the `if engine-from-source` branch around `build.sh`. -- Set `env: ARCADEDB_TAG: ${{ needs.download-jars.outputs.arcadedb-tag }}`. - -- [ ] **Step 2: Add the config files** - -| File | Contents | -|---|---| -| `.github/actionlint.yaml` | The two workflow entries copied from upstream `.github/actionlint.yaml:20-25`. | -| `.github/dependabot.yml` | `version: 2`. Upstream's `uv` entry with `directory: "/"`, plus a weekly `github-actions` entry. Keep the upstream cooldown values for the uv entry. | -| `.pre-commit-config.yaml` | The upstream black, isort (`--profile black`), pretty-format-yaml (`--autofix --indent=2`) and shfmt hooks, with the same `rev`s. Each `files: ^bindings/python/...` regex becomes the same regex without the prefix. | -| `REUSE.toml` | `version = 1`, plus the upstream bindings annotation with paths rewritten to the root: `**/*.py`, `**/*.sh`, `scripts/Dockerfile.build`, `scripts/jar_exclusions.txt`, `pyproject.toml`, `setup.py`, `tests/fixtures/**`. | - -- [ ] **Step 3: Verify** - -Run each of these: -- `actionlint`. Expected: no errors. Install it with `brew install actionlint` if needed. -- `pre-commit run --all-files`. Expected: passes. If the yaml and shfmt autofix hooks rewrite moved files, commit those rewrites as a separate `style:` commit so the history stays readable. -- `git grep -n 'bindings/python\|engine-from-source\|mvnw\|pom.xml' .github`. Expected: no matches. - -- [ ] **Step 4: Commit** — `ci: run the bindings workflows standalone against arcadedata/arcadedb:latest` - ---- - -### Task 6: README and remaining references - -**Files:** -- Modify: `README.md`, `pyproject.toml:6` (comment only) - -- [ ] **Step 1: Update the docs** - -Changes: -- **`README.md`:** drop the `cd bindings/python/` lines. Replace line 138's "extracted from the main ArcadeDB `pom.xml`" with: "taken from the `arcadedb-engine-.jar` in the `arcadedata/arcadedb` image (`latest` by default, set `ARCADEDB_TAG` to pin)". Add a short "Origin" section: "Moved from ArcadeData/arcadedb `bindings/python` at ``, with history." -- **`pyproject.toml:6`:** say "from the bundled engine jar". Leave the URLs and authors unchanged. - -- [ ] **Step 2: Verify** - -Run: `git grep -n -E 'bindings/python|pom\.xml' -- . ':!docs/superpowers'`. Expected: only `tests/test_check_test_skips.py:53`. - -- [ ] **Step 3: Commit** — `docs: README for the standalone repository` - ---- - -### Task 7: Push, CI, merge - -- [ ] **Step 1: Push and open the PR** - -Run `git push -u origin move-python-bindings`. Open a PR to `main` titled "Move Python bindings from ArcadeData/arcadedb (with history)". The body must state `EXTRACTED_SHA`, the freeze, and that the PR must be merged with a merge commit. End the body with `🤖 Generated with [Claude Code](https://claude.com/claude-code)`. - -- [ ] **Step 2: Wait for both workflows to go green** - -Run: `gh pr checks --watch`. Expected: every job passes, including the macOS and Windows native builds. When a job fails, use superpowers:systematic-debugging. Do not add skips, because `check_test_skips.py` forbids them. - -- [ ] **Step 3: Merge** - -Ask the user to confirm first. Then run `gh pr merge --merge`, never `--squash`. Afterwards, `git log --oneline origin/main | wc -l` must be at least 63. - -- [ ] **Step 4: Prove that `main` and the manual trigger work** - -Run: `gh workflow run test-python-bindings.yml --ref main && gh workflow run test-python-examples.yml --ref main`, then watch both with `gh run watch`. Expected: both green. - ---- - -### Task 8: Upstream cleanup PR on ArcadeData/arcadedb - -Start only after Task 7 step 4 is green. Opening this PR needs the user's go-ahead. - -**Files (upstream):** -- Delete: `bindings/python/`, `.github/workflows/test-python-bindings.yml`, `.github/workflows/test-python-examples.yml` -- Modify: - - `.github/workflows/mvn-test.yml`: the `bindings/python/**` paths-ignore entries. - - `.github/workflows/mvn-deploy.yml`: its `paths-ignore` entry. - - `.github/workflows/meterian.yml`: `--exclude-folders`, keeping `**/e2e-go`; trim the comment's bindings sentence. - - `.github/dependabot.yml`: the "Bindings Python" uv entry. - - `.github/actionlint.yaml`: the two entries. - - `.codacy.yml`: `bindings/python/tests/**` and the comment reference. - - `REUSE.toml`: the bindings annotation. - - `.pre-commit-config.yaml`: the four bindings hooks and `bindings/python/mkdocs\.yml` in the yaml exclude. - - `README.md`: point to `ArcadeData/arcadedb-python-bindings`. -- Check and edit if they match: `.github/scripts/check-gitignore-src-paths.sh`, `sbom.json`, root `.gitignore`. -- Leave alone: `e2e-python/`, `docs/**` (historical). - -- [ ] **Step 1: Check the freeze** - -Run, in a fresh worktree of upstream on a branch `chore/remove-python-bindings`: `git fetch origin && git log --oneline ..origin/main -- bindings/python`. - -Expected: empty. If it is not empty, **stop**, port those commits here first, and tell the user. - -- [ ] **Step 2: Make the deletions and edits listed above** - -- [ ] **Step 3: Verify** - -Run: `git grep -n -E 'bindings/python|arcadedb_embedded|arcadedb-embedded' -- . ':!docs' ':!e2e-python'`. Expected: no matches, except the README link to the new repo. - -Also run `pre-commit run --all-files` and `actionlint`. Expected: pass. - -- [ ] **Step 4: Commit, push and open the PR** - -The PR body states `EXTRACTED_SHA`, links the merged PR here, and ends with the Claude Code line. Then wait for upstream CI to go green with `gh pr checks --watch`. Merging is left to the user. diff --git a/docs/superpowers/specs/2026-10-22-python-bindings-move-design.md b/docs/superpowers/specs/2026-10-22-python-bindings-move-design.md deleted file mode 100644 index 728b9200..00000000 --- a/docs/superpowers/specs/2026-10-22-python-bindings-move-design.md +++ /dev/null @@ -1,163 +0,0 @@ -# Move the Python bindings out of the ArcadeDB monorepo - -Date: 2026-10-22 -Source: `ArcadeData/arcadedb` @ `ee04bd202abce0183c6263a3c4d8a61a4d526960`, directory `bindings/python` -Target: `ArcadeData/arcadedb-python-bindings` (this repo; currently only `.gitignore`, `LICENSE`, `README.md`) - -## Intent - -The Python bindings (`arcadedb-embedded`, a JPype wrapper that bundles the ArcadeDB jars and a -JRE into a platform wheel) live in `bindings/python` of the main ArcadeDB repo. They get their own -repo so they build, test and evolve independently of the Java monorepo, and the monorepo stops -carrying Python CI. - -The move is done in three stages, and each stage gates the next: - -1. Move the code (with history) and CI into this repo. -2. Prove it works here: wheel builds and the test and example workflows go green on GitHub Actions. -3. Only then remove `bindings/python` and every reference to it from upstream. - -## Goals - -- All tracked files under `bindings/python` arrive here at the repo root, with their git history. -- The two Python workflows (`test-python-bindings.yml`, `test-python-examples.yml`) arrive and pass here. -- The build no longer depends on the monorepo: the ArcadeDB version comes from the - `arcadedata/arcadedb:latest` Docker image, not from the parent `pom.xml`. -- The bindings follow upstream snapshots automatically. `latest` is republished together with each - `X.Y.Z-SNAPSHOT` tag (verified 2026-10-22: `latest` and `26.11.1-SNAPSHOT` pushed at the same time), - so when upstream moves to the next snapshot, the next build here picks it up with no edit. -- Upstream is cleaned up only after this repo is green. - -## Non-goals - -- The `humemai/arcadedb-embedded-python` repo. Its owner will deal with it later. Keep the - `pyproject.toml` URLs and author fields as they are. -- PyPI publishing. Upstream has no publishing workflow today, and none is added here. -- Behaviour changes to the bindings, tests or examples. -- `e2e-python/` and the `python-e2e-tests` job in upstream `mvn-test.yml`. They test the server over - the wire, not these bindings, so they stay upstream. - -## Anti-goals - -- Deleting anything upstream before this repo's CI is green on `main`. -- Losing history. `git blame` must keep working, because many comments in tests and config explain - past failures by date and issue number. -- Hard-coding a version that someone has to bump by hand on every snapshot. - -## Constraints and current coupling (what has to change) - -- **Version**: `scripts/extract_version.py` and `scripts/write_version.py` read the parent `pom.xml` - (`X.Y.Z-SNAPSHOT` → `X.Y.Z.devN`). `scripts/build.sh` uses it to choose the Docker tag. - `scripts/Dockerfile.build` does `COPY pom.xml /arcadedb/pom.xml`. -- **Paths**: `Dockerfile.build` uses the monorepo root as the build context (`build.sh` passes `../..`) - and copies `bindings/python/...`. Both workflows `cd bindings/python`, and both filter on - `bindings/python/**` (26 occurrences in `test-python-bindings.yml` alone). -- **Engine from source**: the `engine-from-source` input in both workflows builds the jars from the - same commit with `./mvnw -pl package`. This can't work outside the monorepo. -- **Triggers**: `test-python-bindings.yml` has a `workflow_run` on upstream's `Release` workflow, - which doesn't exist here. -- No repository secrets are used by either workflow (only `upload-artifact`). -- The `[tool.uv]` comment in `pyproject.toml` mentions a "repo-root pyproject.toml" that doesn't exist upstream either. -- `git-filter-repo` isn't installed locally (`brew install git-filter-repo`). - -## Approaches considered - -1. **Move with history via `git filter-repo`.** **Chosen.** Filter a fresh clone of upstream down to - `bindings/python/**` plus the two workflow files, renaming `bindings/python/` to the root, then merge - into this repo's `main` with `--allow-unrelated-histories`. Keeps the 63 commits that touched the - bindings and lets the code live at the repo root. Costs a one-off filtering step and requires - rewriting the `bindings/python` paths. -2. **Plain copy.** One import commit referencing the upstream SHA. Simpler, but history and blame - stay behind in upstream. -3. **Plain copy into `bindings/python/`.** Fewest path edits, but leaves a pointless nested folder in a - single-purpose repo. - -The user picked 1. - -## Decisions - -Decided by the user: - -- Move what upstream has now. humemai integration comes later from its owner. -- Bind to the current snapshot and follow the next snapshot automatically, using the `latest` Docker image. -- Move with history (approach 1). - -Calls made while writing this document (Claude's, open to correction): - -- **Layout**: bindings content at the repo root (`src/`, `tests/`, `examples/`, `scripts/`, - `pyproject.toml`, ...). The workflows go to `.github/workflows/`. -- **Version source**: `ARCADEDB_TAG` defaults to `latest` and can be overridden by environment or - build arg (for example `26.11.1-SNAPSHOT` to pin). The Maven version is read from the engine jar - name inside the image (`/home/arcadedb/lib/arcadedb-engine-.jar`), then put through the existing - `X.Y.Z-SNAPSHOT → X.Y.Z.devN` / release conversion. The `pom.xml` reading code is replaced, - not kept as a fallback. -- **Docker build context** becomes this repo's root. The `COPY pom.xml` line is removed. -- **`engine-from-source` input** is removed from both workflows. Its replacement is the - `ARCADEDB_TAG` override or a local jars directory (`build.sh ... /path/to/jars` already supports one). -- **Triggers**: push and PR (no path filter needed in a single-purpose repo), `workflow_dispatch`, plus - a nightly `schedule` so a new snapshot is exercised even when nothing here changes. The - `workflow_run: Release` trigger is dropped. `workflow_call` is kept. -- **Repo hygiene that moves with the code**: the bindings' own `.gitignore` merges into the root - `.gitignore`. The upstream config entries that apply to the bindings come here: the Dependabot - `uv` entry → `.github/dependabot.yml` (directory `/`), the black/isort/yaml pre-commit hooks - → `.pre-commit-config.yaml`, and the bindings line of `REUSE.toml`. - Codacy and Meterian config are not carried over (no such integrations on this repo yet). -- **Test fixtures**: `tests/test_import_database.py` reads five files from upstream's - `integration/src/test/resources/` via `parents[3]`. These are `importer-vertices.csv`, - `importer-edges.csv`, `neo4j-export-mini.jsonl`, `importer-word2vec.txt` and - `importer-rdf.xml`, about 10 KB in total. Copy them into `tests/fixtures/` and point the path - there. This is a path change, not a behaviour change. Skipping the tests isn't an option: - `scripts/check_test_skips.py` fails CI on any skip. Examples 07–12 get the same treatment for - `parents[3]` and `bindings/python/dist`. -- **Issue references in imported history**: the `filter-repo` step rewrites `#NNNN` in commit - messages to `ArcadeData/arcadedb#NNNN`, so the links keep pointing at upstream issues. This must - be done before the first push. -- **`latest` vs. local-jar check**: `build.sh` compares the Docker tag to the jar version. With - `latest`, it must compare against the version read from the jars instead. -- **Also carried over**: the fourth upstream pre-commit hook (shfmt on the bindings' `.sh`) and - the `.github/actionlint.yaml` entries for both workflows. The root `.gitignore` here ignores - `lib/`, which must not hide `local-jars/lib/` handling. -- **README**: the root README becomes the bindings README, plus a short note on where the code - came from and how the version is chosen. -- **How it lands here**: the history merge is pushed to a branch and goes in via a PR to `main`, so CI - runs before `main` changes. The PR is opened only after a local `scripts/build.sh` build plus its - tests pass. - -## Freeze - -Decided by the user: `bindings/python` is frozen upstream from the moment the history is extracted -until Stage 3 deletes it. The extraction uses upstream `main` as of that moment, and the SHA in the -header of this document is updated to match. There's no re-sync step. - -Stage 3 starts by running `git log ..origin/main -- bindings/python`. It must be -empty. If anything landed despite the freeze, stop and port it here before deleting anything. -The Stage 3 PR states the extracted SHA. - -## Stage 3: upstream cleanup (after green `main` here) - -One PR on `ArcadeData/arcadedb` that: - -- deletes `bindings/python/`, `.github/workflows/test-python-bindings.yml` and `test-python-examples.yml`; -- removes the `bindings/python` entries from `mvn-test.yml` and `mvn-deploy.yml` (`paths-ignore`), `meterian.yml` - (`--exclude-folders`), `dependabot.yml`, `.codacy.yml`, `REUSE.toml` and `.pre-commit-config.yaml` - (the three hooks filtered to `bindings/python` and the yaml exclude); -- checks `.github/scripts/check-gitignore-src-paths.sh`, `README.md` and `sbom.json` for references and - points the README at the new repo; -- does not touch `e2e-python/` or historical docs under `docs/`. - -## Success criteria - -- `main` here has the bindings at the root, and `git log --follow src/arcadedb_embedded/` shows - upstream commits. -- `scripts/build.sh` builds a wheel locally against `latest` with no ArcadeDB checkout present, and the - wheel's version matches the image's snapshot (today `26.11.1.devN`). -- Both workflows pass on `main` here, including a manual `workflow_dispatch` run. -- Upstream cleanup PR is merged with upstream CI green. - -## Left to the builder - -- Exact `git filter-repo` invocation and how the two workflow files are pulled in alongside the directory rename. -- How the `.devN` counter behaves without `pom.xml`. Keep the existing tracker logic if it works, otherwise - pick something deterministic (for example a build number) and note it. -- Workflow job and matrix contents beyond the path, trigger and `engine-from-source` changes above. -- The order of commits within the PR. diff --git a/scripts/check_no_embedded_name.py b/scripts/check_no_embedded_name.py index d9ddbcd3..9a7ab242 100644 --- a/scripts/check_no_embedded_name.py +++ b/scripts/check_no_embedded_name.py @@ -13,8 +13,7 @@ What the allowlist lets through, and why: -* Whole files that are history or that must name the old package to do their job: the plan and spec - documents under docs/superpowers/, a changelog, and this script. +* Whole files that are history or that must name the old package to do their job: a changelog, and this script. * References to issues of the old repository, written `humemai/arcadedb-embedded-python#123`. They point at real threads, so they keep their text. * The sentence "This package was published as arcadedb-embedded until 26.10.x; install arcadedb or arcadedb-jvm now." @@ -48,7 +47,6 @@ # Files (fnmatch patterns on the path relative to the repository root) that may name the old package. ALLOWED_PATHS: tuple[str, ...] = ( - "docs/superpowers/*", # plans and specs of the move, history "CHANGELOG*", "HISTORY*", "NEWS*", diff --git a/tests/conftest.py b/tests/conftest.py index ec3147c7..0ae9b365 100644 --- a/tests/conftest.py +++ b/tests/conftest.py @@ -87,7 +87,7 @@ os.path.join(os.path.dirname(os.path.dirname(__file__)), "docs", "index.md") ): # the user documentation site (docs/index.md and its pages) is not in this repository; - # docs/ alone is not enough, it also holds design notes (docs/superpowers) + # docs/ holds the Antora component (docs/modules/ROOT), not that site collect_ignore.append("test_docs_examples.py")