Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 42 additions & 7 deletions .github/workflows/deploy-pages.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,17 @@
name: Deploy to GitHub Pages

on:
push:
branches: [main]
paths:
- "astro-site/**"
- "compressionkit/**"
- "examples/**"
- "configs/**"
- "scripts/**"
- "pyproject.toml"
- "uv.lock"
- ".github/workflows/deploy-pages.yml"
workflow_dispatch:
workflow_call:

Expand Down Expand Up @@ -39,18 +50,42 @@ jobs:
with:
enable-cache: true

- name: Install and Build
env:
CI: ""
PUBLIC_URL: "${{ steps.pages.outputs.base_url }}/"
- name: Set up Node
uses: actions/setup-node@v4
with:
node-version: '24'
cache: npm
cache-dependency-path: astro-site/package-lock.json

- name: Install documentation dependencies
working-directory: astro-site
run: |
npm ci
npx playwright install --with-deps chromium

- name: Validate and build documentation
working-directory: astro-site
run: |
npm run check
npm run check:content
npm run build
npm run check:links
npm run check:migration

- name: Check rendered documentation
working-directory: astro-site
run: |
uv sync --only-group docs
uv run zensical build
npm exec astro preview -- --host 127.0.0.1 --port 8878 > /tmp/astro-preview.log 2>&1 &
for attempt in $(seq 1 30); do
curl -fsS http://127.0.0.1:8878/compressionkit/ > /dev/null && break
sleep 1
done
DOCS_AUDIT_URL=http://127.0.0.1:8878 npm run check:layout

- name: Upload artifact
uses: actions/upload-pages-artifact@v3
with:
path: ./site
path: ./astro-site/dist

deploy:
environment:
Expand Down
53 changes: 53 additions & 0 deletions .github/workflows/docs-preview.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
name: Astro documentation checks
on:
pull_request:
paths:
- 'astro-site/**'
- 'pyproject.toml'
- 'uv.lock'
- 'compressionkit/**'
- 'examples/**'
- 'configs/**'
- 'scripts/**'
- '.github/workflows/docs-preview.yml'
- '.github/workflows/deploy-pages.yml'
workflow_dispatch:
permissions:
contents: read
jobs:
docs:
runs-on: ubuntu-latest
defaults:
run:
working-directory: astro-site
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: '24'
cache: npm
cache-dependency-path: astro-site/package-lock.json
- uses: actions/setup-python@v5
with:
python-version: '3.12'
- uses: astral-sh/setup-uv@v5
- run: npm ci
- name: Install browser for rendered checks
run: npx playwright install --with-deps chromium
- run: npm run check
- run: npm run check:content
- run: npm run build
- run: npm run check:links
- run: npm run check:migration
- name: Check rendered documentation
run: |
npm exec astro preview -- --host 127.0.0.1 --port 8878 > /tmp/astro-preview.log 2>&1 &
for attempt in $(seq 1 30); do
curl -fsS http://127.0.0.1:8878/compressionkit/ > /dev/null && break
sleep 1
done
DOCS_AUDIT_URL=http://127.0.0.1:8878 npm run check:layout
- uses: actions/upload-artifact@v4
with:
name: compressionkit-astro-preview
path: astro-site/dist
11 changes: 11 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -197,3 +197,14 @@ internal/**
# Local-only model weight workspace (large binaries; published artifacts go to
# results/ or HuggingFace).
/models/

# Astro generated output
astro-site/node_modules/
astro-site/dist/
astro-site/.astro/
astro-site/.cache/
astro-site/src/data/
astro-site/src/content/docs/reference/
astro-site/public/reference/
astro-site/src/content/docs/guides/
astro-site/public/notebooks/
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -28,5 +28,5 @@ exclude: |
site/.*|
slides/.*|
results/.*|
docs/assets/plots/.*
astro-site/public/assets/plots/.*
)$
6 changes: 3 additions & 3 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ These are the project guidelines for agentic AI models working in this repositor
- For logging and metrics, prefer TensorBoard and Keras callbacks where possible.

## Experiment Architecture
Three layers, kept deliberately separate (see `docs/experiment-architecture.md` for the full rationale):
Three layers, kept deliberately separate (see `astro-site/src/content/docs/experiment-architecture.md` for the full rationale):
1. **Blocks** — small, importable capabilities (datasets/cache builders, preprocessing/augmentation, model builders, losses/metrics/callbacks, scorecard builders, exporters/validators). No hidden global state, no mandatory registries.
2. **Ready-made experiments** — readable end-to-end flows built from blocks. Opinionated but never the only path.
3. **Golden releases** — the release boundary: stable IDs, reproducible configs, scorecards, deploy packages. An experiment becomes golden by passing the artifact contract, not by being rewritten around a mandatory base class.
Expand All @@ -41,7 +41,7 @@ Three layers, kept deliberately separate (see `docs/experiment-architecture.md`
- When a shared trainer or runner starts accumulating export, validation, or scorecard policy, extract that behavior into standalone helpers before adding new abstract hooks.
- Consider moving generic Keras 3 edge-model training/deployment blocks to HeliaEdge once they are no longer compressionKIT-specific. Good candidates include RVQ/VQ layers, reusable RVQ architectures, generic losses/metrics/callbacks, LiteRT export helpers, and reference-vector utilities.
- Keep modality-specific datasets, physiological scorecards, signal preprocessing policy, golden registry entries, HuggingFace naming, and v1 release policy in compressionKIT.
- Entropy-coding research (new priors/entropy coders as a second stage on top of a frozen codec) follows this same pattern; see `docs/entropy-coding-research.md` for the current blocks, the promotion path, and known lessons (don't re-derive these).
- Entropy-coding research (new priors/entropy coders as a second stage on top of a frozen codec) follows this same pattern; see `maintainer-notes/entropy-coding-research.md` for the current blocks, the promotion path, and known lessons (don't re-derive these).

## Style
- Docstrings should use Google style.
Expand Down Expand Up @@ -73,7 +73,7 @@ Three layers, kept deliberately separate (see `docs/experiment-architecture.md`
- Customers will see this codebase — keep public-facing code, configs, and docs clean.
- Golden run naming: `{modality}_rvq_{sample_rate}hz_{cr}x_golden/` (e.g., `ppg_rvq_64hz_04x_golden/`). Runs live under `results/` (git-ignored) — commit only configs, scripts, and code; never model weights or large artifacts.
- Deploy artifacts are exported via `compressionkit.export.deploy.export_for_deployment()` into each run's `deploy/` subdirectory (`.tflite`, `.h`, `.npz`, `.json`) — the only publishable outputs.
- Codec-family dispatch (loader class, required files, model card, default license) is centralized in `compressionkit.export.family_registry.FAMILY_REGISTRY` — adding a new codec family (`rvq`/`spiht`/`hybrid` today) means adding one entry there, not editing the runtime loader, validator, and publisher independently. See `docs/adding-a-codec-family.md` for the full concept-to-release lifecycle.
- Codec-family dispatch (loader class, required files, model card, default license) is centralized in `compressionkit.export.family_registry.FAMILY_REGISTRY` — adding a new codec family (`rvq`/`spiht`/`hybrid` today) means adding one entry there, not editing the runtime loader, validator, and publisher independently. See `astro-site/src/content/docs/adding-a-codec-family.md` for the full concept-to-release lifecycle.
- Before cutting a release, run `compressionkit golden validate-all --strict-release` to audit every registered golden's existing local deploy package for schema drift (gitignored `results/` can silently go stale relative to code changes).

### HuggingFace Releases
Expand Down
14 changes: 8 additions & 6 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ workflow, code style, and how to add new signals, models, or configs.

- Python 3.12
- [`uv`](https://docs.astral.sh/uv/) for project and package management
- Node.js 24 and npm 11 for documentation
- (Optional) NVIDIA GPU + CUDA for training — a dev container with CUDA is provided under `.devcontainer/`

## Getting set up
Expand All @@ -15,7 +16,7 @@ workflow, code style, and how to add new signals, models, or configs.
git clone https://github.com/AmbiqAI/compressionkit.git
cd compressionkit
uv sync --group dev # runtime + dev deps
uv sync --group docs # (optional) docs build deps
npm --prefix astro-site ci # docs dependencies; requires Node 24
uv run pre-commit install # install git hooks
```

Expand All @@ -35,8 +36,9 @@ uv run ruff format .
uv run pre-commit run --all-files

# Build the docs locally
uv run --group docs zensical serve # live-reload at http://localhost:8000
uv run --group docs zensical build # static site in site/
npm --prefix astro-site ci # Node 24
npm --prefix astro-site run dev # local preview
npm --prefix astro-site run build # static site in astro-site/dist/
```

## Code style
Expand Down Expand Up @@ -87,9 +89,9 @@ Ruff enforces the rest. The config lives in [`pyproject.toml`](pyproject.toml).

## Documentation

- Public docs live under [`docs/`](docs/) and are served via Zensical / MkDocs Material.
- If you add a new CLI command, config, or model family, add a page (or extend an existing one) and update the nav in [`zensical.toml`](zensical.toml).
- API docs are generated via `mkdocstrings` — a well-formed docstring is enough, no extra wiring needed.
- Public docs live under [`astro-site/src/content/docs/`](astro-site/src/content/docs/) and are built with Astro/Starlight.
- If you add a new CLI command, config, or model family, add a page (or extend an existing one) and update the nav in [`astro-site/src/navigation.mjs`](astro-site/src/navigation.mjs).
- API docs use static Griffe extraction. Add public modules to `astro-site/scripts/public-api.json` and maintain their docstrings.

## Pull requests

Expand Down
5 changes: 3 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -140,7 +140,7 @@ compressionkit/ # Importable Python package
└── synthetic/ # Synthetic signal and artifact generation

configs/ # Versioned YAML configs
docs/ # Documentation site
astro-site/ # Astro documentation site
examples/ # Notebook quickstarts
scripts/ # Release, evaluation, and plotting utilities
tests/ # pytest suite
Expand All @@ -155,7 +155,8 @@ uv sync --group dev --extra hf
uv run ruff check compressionkit/ tests/ scripts/
uv run ruff format compressionkit/ tests/ scripts/
uv run pytest tests/ -ra
./.venv/bin/python -m zensical build
npm --prefix astro-site ci
npm --prefix astro-site run build
```

---
Expand Down
1 change: 1 addition & 0 deletions astro-site/.nvmrc
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
24
Loading
Loading