feat(loqe): read ADLS warehouses via the azure_wasm community extension

[HynekBlaha]

DuckDB-WASM has no Azure filesystem: the official azure extension has no wasm build, so LoQE could not read abfss:// tables and ADLS warehouses were excluded from the storage whitelist.

azure_wasm (https://community-extensions.duckdb.org, source: HynekBlaha/duckdb-azure-wasm) is a loadable wasm-only extension that claims the abfss/abfs/az/wasbs schemes, registers the azure secret type and rewrites requests to https://<host>/<container>/<key>?<sas>. That is exactly the shape DuckDB's iceberg extension produces from Lakekeeper's vended adls.sas-token.<account>.<suffix> credentials (account_name + AccountName=...;SharedAccessSignature=...), so no fork of duckdb-wasm is needed.

It is only published for DuckDB core v1.5.5, hence the duckdb-wasm bump:

• pin @duckdb/duckdb-wasm to 1.33.1-dev64.0 (core v1.5.5; the latest dist-tag still predates the v1.5.5 bump, and a caret range would not match 1.33.1-dev*)
• vendor extensions for v1.5.5 and pull azure_wasm from the community repository, which uses the same /v<ver>/<platform>/ layout, so the air-gapped mirror keeps working unchanged
• LOAD azure_wasm before the first catalog scan, otherwise DuckDB tries to autoload the official azure extension and 404s against the mirror. Best-effort: s3/gcs warehouses do not need it
• ATTACH option is STAGE_CREATE_TABLES since iceberg 1.5.x; the old SUPPORT_STAGE_CREATE is silently ignored, which disabled staged creates
• allow adls in the DuckDB-WASM storage whitelist

The lockfile also gains duckdb-wasm's new qs dependency tree.

Verified against the published binaries: core reports v1.5.5 / wasm_eh, INSTALL + LOAD azure_wasm succeed, azure shows up in duckdb_secret_types() and abfss:// paths reach the extension.

Tested complete integration with local lakekeeper fork build:

Summary by CodeRabbit

• New Features

  • Added support for connecting to Azure Data Lake Storage (ADLS) using SAS-token authentication.
  • Added compatibility for Azure-specific storage extensions while preserving support for other catalog types.

• Bug Fixes

  • Updated Iceberg catalog attachment behavior for compatibility with the latest DuckDB release.
  • Improved catalog connections so unavailable optional Azure functionality does not prevent non-Azure catalogs from working.

[coderabbitai]

Walkthrough

This change updates DuckDB WASM and extension versions, adds community extension downloads for azure_wasm, enables ADLS validation and runtime loading, and replaces the Iceberg catalog attachment option with STAGE_CREATE_TABLES.

Changes

DuckDB and ADLS integration

Layer / File(s)	Summary
DuckDB extension packaging package.json, scripts/download-duckdb-extensions.mjs, src/composables/loqe/LoQEEngine.ts	Pins DuckDB WASM to 1.33.1-dev64.0, updates extensions to v1.5.5, and downloads azure_wasm from the community repository.
ADLS validation and runtime loading src/composables/useStorageValidation.ts, src/composables/loqe/LoQEEngine.ts	Adds adls storage support and loads azure_wasm before Iceberg, while continuing when the community extension is unavailable.
Iceberg attachment option migration src/composables/loqe/CatalogManager.ts	Uses STAGE_CREATE_TABLES true for initial catalog attachment and reattachment flows.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Merge Risk: 🟡 Moderate · up to 58d0f

This change adds ADLS warehouse support and updates Iceberg staging behavior, but an azure_wasm loading failure may leave an ADLS catalog appearing available while queries fail later, and the staged-create option still needs confirmation against the pinned runtime. Merge should wait for targeted verification or explicit owner acceptance.

Sequence Diagram(s)

sequenceDiagram
  participant attachCatalog
  participant azure_wasm
  participant Iceberg
  attachCatalog->>azure_wasm: Install and load extension
  azure_wasm-->>attachCatalog: Return loaded or unavailable
  attachCatalog->>Iceberg: Load and attach catalog
  Iceberg-->>attachCatalog: Return attachment result

Loading

Possibly related PRs

• lakekeeper/console-components#89: Updates DuckDB WASM dependencies and related assets.
• lakekeeper/console-components#228: Changes LoQEEngine.ts extension repository handling.
• lakekeeper/console-components#231: Updates pinned DuckDB extension downloads.

Suggested reviewers: v-kessler

Poem

I’m a rabbit with Azure in sight,
DuckDB extensions now load just right.
Catalog stages hop into place,
SAS tokens guide the data chase.
azure_wasm joins the burrow tonight.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely describes the main change: enabling LoQE to read ADLS warehouses through the azure_wasm community extension.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[cla-assistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[HynekBlaha]

@coderabbitai review

[coderabbitai]

✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

[coderabbitai]

🔇 Additional comments (5)

package.json (1)

90-90: LGTM!

scripts/download-duckdb-extensions.mjs (1)

17-20: LGTM!

Also applies to: 39-48, 89-98, 108-108

src/composables/loqe/LoQEEngine.ts (1)

598-603: 🎯 Functional Correctness | 🟠 Major | 🏗️ Heavy lift

⚠️ Unverified finding
Sandbox verification was unavailable.

Do not silently accept an ADLS catalog when azure_wasm fails.

useStorageValidation.ts now marks adls as supported, but this catch only logs the error and lets attachCatalog resolve. An ADLS query can then fail later with no user-facing feedback. Route the error through handleError(e, context, notify?). If the catalog is ADLS, propagate the failure or mark the catalog unavailable. Keep the fallback only for non-ADLS catalogs.

As per coding guidelines, always use handleError(error, context, notify?) for error handling in components and API functions.

src/composables/useStorageValidation.ts (1)

16-19: LGTM!

src/composables/loqe/CatalogManager.ts (1)

52-60: 📐 Maintainability & Code Quality

Verify the option behavior against the exact vendored Iceberg build.

The current DuckDB REST-catalog documentation lists STAGE_CREATE_TABLES. However, a recent DuckDB-WASM report shows support_stage_create producing an “Unhandled options found” error. Run the initial attach and both reattach flows against the pinned 1.33.1-dev64.0 WASM bundle and vendored v1.5.5 extension. Reword the comment if unknown options are rejected rather than silently collected. (duckdb.org)

Also applies to: 159-159, 231-231

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 10e5bcda-60f5-4e1d-aad4-299c61a597e9

📥 Commits

Reviewing files that changed from the base of the PR and between ebcd080 and 58d0fd0.

⛔ Files ignored due to path filters (1)

• package-lock.json is excluded by !**/package-lock.json

📒 Files selected for processing (5)

• package.json
• scripts/download-duckdb-extensions.mjs
• src/composables/loqe/CatalogManager.ts
• src/composables/loqe/LoQEEngine.ts
• src/composables/useStorageValidation.ts

Included review availability: Your plan includes up to 1 review per rolling hour; 0 remain after this review.