chore: merge new changes from ipfs/kubo master - #2
Open
alvin-reyes wants to merge 972 commits into
Open
Conversation
Gracefully shutdown the ipfs node to ensure data is saved. Forceful shutdown is done if there is an error sending the interrupt signal to the process, such as on Windows.
* datastore: upgrade go-ds-flatfs to v0.6.0 See: ipfs/go-ds-flatfs#142 * docs(changelog): add go-ds-flatfs atomic batch writes *documents the new flatfs batch implementation that uses atomic operations via temp directory, preventing orphan blocks on interrupted imports and reducing memory usage. * includes improved tests, batch cleanup fixes, and docs * docs(changelog): reframe go-ds-flatfs entry for users focus on user benefits instead of implementation details
Co-authored-by: Marcin Rataj <lidel@lidel.org>
Fix panic when broken link is created in watch directory. Closes #10017
* feat(provider): log fullrt crawl * fix(provider): improve AcceleratedDHTClient log messages use config option name instead of internal "fullrt" jargon, align terminology with daemon startup message, add time estimate --------- Co-authored-by: Marcin Rataj <lidel@lidel.org>
* feat(config): add Gateway.MaxRequestDuration option exposes the previously hardcoded 1 hour gateway request deadline as a configurable option, allowing operators to adjust it to fit deployment needs. protects gateway from edge cases and slow client attacks. boxo: ipfs/boxo#1079 * test(gateway): add MaxRequestDuration integration test verifies config is wired correctly and 504 is returned when exceeded * docs: add MaxRequestDuration to gateway production guide --------- Co-authored-by: Andrew Gillis <11790789+gammazero@users.noreply.github.com>
* feat(pubsub): persistent seqno validation and diagnostic commands - upgrade go-libp2p-pubsub to v0.15.0 - add persistent seqno validator using BasicSeqnoValidator stores max seen seqno per peer at /pubsub/seqno/<peerid> survives daemon restarts, addresses message cycling in large networks (#9665) - add `ipfs pubsub reset` command to clear validator state - add `ipfs diag datastore get/count` commands for datastore inspection requires daemon to be stopped, useful for debugging - change pubsub status from Deprecated to Experimental - add CLI tests for pubsub and diag datastore commands - remove flaky pubsub_msg_seen_cache_test.go (replaced by CLI tests) * fix(pubsub): improve reset command and add deprecation warnings - use batched delete for efficient bulk reset - check key existence before reporting deleted count - sync datastore after deletions to ensure persistence - show "no validator state found" when resetting non-existent peer - log deprecation warnings when using --enable-pubsub-experiment or --enable-namesys-pubsub CLI flags * refactor(test): add datastore helpers to test harness --------- Co-authored-by: Andrew Gillis <11790789+gammazero@users.noreply.github.com>
* Implements the -l/--long flag for the ipfs ls command to display Unix-style file permissions and modification times, similar to the traditional ls -l. When the --long flag is used, the output includes: - File mode/permissions in Unix format (e.g., -rw-r--r--, drwxr-xr-x) - File hash (CID) - File size (when --size is also specified) - Modification time in human-readable format - File name The permission string implementation handles all file types and special bits: - File types: regular (-), directory (d), symlink (l), named pipe (p), socket (s), character device (c), block device (b) - Special permission bits: setuid (s/S), setgid (s/S), sticky (t/T) - Lowercase when execute bit is set, uppercase when not set The timestamp format follows Unix ls conventions: - Recent files (within 6 months): "Jan 02 15:04" - Older files: "Jan 02 2006" Signed-off-by: sneax <paladesh600@gmail.com> * fix(ls): correct --long flag header order and help text - fix header column order: was "Mode Hash Size Name ModTime" but data outputs "Mode Hash Size ModTime Name", now headers match data order - remove redundant if/else branch in directory output that had identical code in both branches - add example output to help text showing format with mode, hash, size, mtime, and name columns - document that files without preserved metadata show '----------' for mode and '-' for mtime - add changelog entry for v0.40 * test(ls): add format stability tests for --long flag add tests to prevent formatting regressions in ipfs ls --long output: unit tests (core/commands/ls_test.go): - TestFormatMode: 20 cases covering all file types (regular, dir, symlink, pipe, socket, block/char devices) and special permission bits (setuid, setgid, sticky with/without execute) - TestFormatModTime: zero time, old time (year format), future time, format length consistency integration tests (test/cli/ls_test.go): - explicit full output comparison with deterministic CIDs to catch any formatting changes - header column order verification for --long with --size=true/false - files without preserved metadata (---------- and - placeholders) - directory output (trailing slash, d prefix in mode) requested in: #11103 (comment) * fix(ls): improve --long flag docs and fix minor issues - improved godocs for formatMode and formatModTime functions - fixed permBit signature: char rune → char byte (avoids unnecessary cast) - clarified help text: mode/mtime are optional UnixFS metadata - documented that times are displayed in UTC - fixed flaky time test by using 1 month ago instead of 1 hour - removed hardcoded CID assertion that would break on DAG changes * fix(ls): show "-" for missing mode in --long output display "-" instead of "----------" when mode metadata is not preserved. this avoids ambiguity with Unix mode 0000 and matches how missing mtime is already displayed. follows common Unix tool conventions (ps, netstat) where "-" indicates "not available". --------- Signed-off-by: sneax <paladesh600@gmail.com> Co-authored-by: Marcin Rataj <lidel@lidel.org>
Reprovide Sweep is now the default behaviour since v0.39.0. Update manual reprovide error message to reflect this change.
experimental setup to surface missed ecosystem updates via monthly PRs groups ipfs, libp2p, multiformats, ipld, and golang.org/x dependencies to reduce noise while ensuring visibility into available updates
* chore: exclude ancient +incompatible versions from go.mod prevents Dependabot from failing when it tries to update go-ipfs-cmds and go-libp2p directly and resolves to pre-Go-modules v2.x/v6.x versions that reference deleted packages * chore(deps): group opentelemetry, prometheus, and uber packages reduces PR noise by batching related observability dependencies * ci: add workflow to run make mod_tidy on Dependabot PRs ensures all go.mod/go.sum files stay in sync when Dependabot updates dependencies in the root module supports manual dispatch with PR number for existing PRs
Bumps [actions/setup-go](https://github.com/actions/setup-go) from 5 to 6. - [Release notes](https://github.com/actions/setup-go/releases) - [Commits](actions/setup-go@v5...v6) --- updated-dependencies: - dependency-name: actions/setup-go dependency-version: '6' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [actions/checkout](https://github.com/actions/checkout) from 4 to 6. - [Release notes](https://github.com/actions/checkout/releases) - [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md) - [Commits](actions/checkout@v4...v6) --- updated-dependencies: - dependency-name: actions/checkout dependency-version: '6' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* fix(routing): update kad-dht with peerstore address clone fix closes #11116 See #11116 for context of this fix * fix(routing): update kad-dht with CPL exploration fix fixes an infinite loop when all peers share the same CPL during provider exploration See libp2p/go-libp2p-kad-dht#1216 * fix(routing): update kad-dht with shutdown loop check libp2p/go-libp2p-kad-dht#1217 * depend on latest kad-dht fix * bump kad-dht to v0.37.0 --------- Co-authored-by: guillaumemichel <guillaume@michel.id> Co-authored-by: Guillaume Michel <guillaumemichel@users.noreply.github.com>
- upgrade go-ds-pebble to [v0.5.9](https://github.com/ipfs/go-ds-pebble/releases/tag/v0.5.9) - git ignore ipfswatch binary
pebble, leveldb, and badger should be updated via go-ds-* wrappers to ensure compatibility
* Upgrade to Boxo v0.36.0 * sharness: add missing metrics
) Bumps [github.com/tidwall/gjson](https://github.com/tidwall/gjson) from 1.16.0 to 1.18.0. - [Commits](tidwall/gjson@v1.16.0...v1.18.0) --- updated-dependencies: - dependency-name: github.com/tidwall/gjson dependency-version: 1.18.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* ci: add caching to ipfs-webui interop tests cache node_modules, Playwright browsers, and test build output to speed up repeated CI runs. also use node version from ipfs-webui/.tool-versions instead of hardcoding, and upload test artifacts on failure. * docs(ci): add header comment to interop workflow explain what helia-interop and ipfs-webui jobs do
* feat(dns): resolve libp2p.direct addresses locally without network I/O p2p-forge hostnames encode IP addresses directly (e.g., 1-2-3-4.peerID.libp2p.direct -> 1.2.3.4), so DNS queries are wasteful. kubo now parses these IPs in-memory. - applies to both default libp2p.direct and custom AutoTLS.DomainSuffix - TXT queries still delegate to network for ACME DNS-01 compatibility - #11140 (comment) use fallback to network DNS instead of returning errors when local parsing fails, ensuring forward compatibility with future DNS records - #11140 (comment) add peerID validation using peer.Decode(), matching libp2p.direct server behavior, with fallback on invalid peerID - #11140 (comment) document interaction with DNS.Resolvers in config.md - #11140 (comment) add AutoTLS.SkipDNSLookup config flag to disable local resolution (useful for debugging or custom DNS override scenarios) - #11140 (comment) add E2E test verifying libp2p.direct resolves locally even when DNS.Resolvers points to a broken server additional improvements: - use madns.BasicResolver interface instead of custom basicResolver - add compile-time interface checks for p2pForgeResolver and madns.Resolver - refactor tests: merge IPv4/IPv6, add helpers, use config.DefaultDomainSuffix - improve changelog to explain public good benefit (reducing DNS load) Fixes #11136
- docs/README.md: restructure to surface 20+ previously undiscoverable docs - docs/README.md: fix broken github-issue-guide.md link (file was removed) - docs/add-code-flow.md: rewrite with current code flow and mermaid diagrams - docs/customizing.md, docs/gateway.md: use specs.ipfs.tech URLs - README.md: fix orphan #nix anchor, use go.dev links, link to contributors graph - remove stale docs/AUTHORS and docs/generate-authors.sh (last updated 2016)
* feat(key): add 'ipfs key ls' as alias for 'ipfs key list' Add 'ls' as an alias for the 'list' subcommand in 'ipfs key' to be consistent with other ipfs commands like 'ipfs repo ls' and 'ipfs pin ls' which use 'ls' instead of 'list'. Fixes #10976 Signed-off-by: Vedant Madane <6527493+VedantMadane@users.noreply.github.com> * feat(key): make 'ipfs key ls' canonical, deprecate 'list' aligns with other commands like 'ipfs pin ls' and 'ipfs files ls'. 'ipfs key list' still works but shows deprecation warning. * fix(key): correct --key option description in verify command was copy-pasted from sign command and said "signing" instead of "verifying" --------- Signed-off-by: Vedant Madane <6527493+VedantMadane@users.noreply.github.com> Co-authored-by: Marcin Rataj <lidel@lidel.org>
* fix http header when compress enabled for get command Closes #2376 * fix(rpc): set Content-Type for ipfs get based on output format - set application/x-tar when outputting tar (default and --archive) - set application/gzip when compression is enabled (--compress) - update go-ipfs-cmds with Tar encoding type and RFC 6713 compliant MIME types (application/gzip instead of application/x-gzip) * test(rpc): add Content-Type header tests for ipfs get * feat(rpc): add Content-Type headers for binary responses set proper Content-Type headers for RPC endpoints that return binary data: - `dag export`: application/vnd.ipld.car - `block get`: application/vnd.ipld.raw - `diag profile`: application/zip - `get`: application/x-tar or application/gzip (already worked, migrated to new API) uses the new OctetStream encoding type and SetContentType() method from go-ipfs-cmds to specify custom MIME types for binary responses. refs: #2376 * feat(rpc): add `ipfs name get` command for IPNS record retrieval add dedicated command to retrieve raw signed IPNS records from the routing system. returns protobuf-encoded IPNS record with Content-Type `application/vnd.ipfs.ipns-record`. this provides a more convenient alternative to `ipfs routing get /ipns/<name>` which returns JSON with base64-encoded data. the raw output can be piped directly to `ipfs name inspect`: ipfs name get <name> | ipfs name inspect spec: https://specs.ipfs.tech/ipns/ipns-record/ * feat(rpc): add `ipfs name put` command for IPNS record storage adds `ipfs name put` to complement `ipfs name get`, allowing users to store IPNS records obtained from external sources without needing the private key. useful for backup, restore, and debugging workflows. the command validates records by default (signature, sequence number). use `--force` to bypass validation for testing how routing handles malformed or outdated records. also reorganizes test/cli files: - rename http_rpc_* -> rpc_* to match existing convention - merge name_get_put_test.go into name_test.go - add file header comments documenting test purposes * chore(deps): update go-ipfs-cmds to latest master includes SetContentType() for dynamic Content-Type headers --------- Co-authored-by: Marcin Rataj <lidel@lidel.org>
…7.0 (#11178) Bumps [github.com/coreos/go-systemd/v22](https://github.com/coreos/go-systemd) from 22.5.0 to 22.7.0. - [Release notes](https://github.com/coreos/go-systemd/releases) - [Commits](coreos/go-systemd@v22.5.0...v22.7.0) --- updated-dependencies: - dependency-name: github.com/coreos/go-systemd/v22 dependency-version: 22.7.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
… to 0.5.0 in the ipfs-ecosystem group (#11177) * chore(deps): bump github.com/multiformats/go-multiaddr-dns Bumps the ipfs-ecosystem group with 1 update: [github.com/multiformats/go-multiaddr-dns](https://github.com/multiformats/go-multiaddr-dns). Updates `github.com/multiformats/go-multiaddr-dns` from 0.4.1 to 0.5.0 - [Release notes](https://github.com/multiformats/go-multiaddr-dns/releases) - [Commits](multiformats/go-multiaddr-dns@v0.4.1...v0.5.0) --- updated-dependencies: - dependency-name: github.com/multiformats/go-multiaddr-dns dependency-version: 0.5.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: ipfs-ecosystem ... Signed-off-by: dependabot[bot] <support@github.com> * chore: run make mod_tidy --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Andrew Gillis <11790789+gammazero@users.noreply.github.com>
…#11160) * chore(deps): bump github.com/hashicorp/go-version from 1.7.0 to 1.8.0 Bumps [github.com/hashicorp/go-version](https://github.com/hashicorp/go-version) from 1.7.0 to 1.8.0. - [Release notes](https://github.com/hashicorp/go-version/releases) - [Changelog](https://github.com/hashicorp/go-version/blob/main/CHANGELOG.md) - [Commits](hashicorp/go-version@v1.7.0...v1.8.0) --- updated-dependencies: - dependency-name: github.com/hashicorp/go-version dependency-version: 1.8.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * chore: run make mod_tidy --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Andrew Gillis <11790789+gammazero@users.noreply.github.com>
* fix: recognize content-type application/x-tar For context, see: ipfs/go-ipfs-cmds#320 Closes: #11179
add missing fixes from v0.47.0 release
* test: IPIP-523 format query precedence over Accept header update boxo to ipfs/boxo#1074 and gateway-conformance tests to ipfs/gateway-conformance#252 for testing IPIP-523 changes where ?format= URL query parameter takes precedence over Accept HTTP header * chore: update boxo and gateway-conformance for IPIP-523 testing - boxo@9aeb0c835899fedb4f886040a4ccf3ba09fd97d4 - gateway-conformance@c82a2a9bc79303e0f07216a80cf454ef2a1e042c * chore(ci): switch to gateway-conformance@v0.9 * chore: update boxo with IPIP-523 changes * chore: update boxo to main after ipfs/boxo#1074 merge * docs: add gateway-conformance v0.9 to changelog
Opens development for the next release after v0.43. - version.go -> 0.44.0-dev - add docs/changelogs/v0.44.md stub and link it in CHANGELOG.md
Bumps [actions/setup-go](https://github.com/actions/setup-go) from 6 to 7. - [Release notes](https://github.com/actions/setup-go/releases) - [Commits](actions/setup-go@v6...v7) --- updated-dependencies: - dependency-name: actions/setup-go dependency-version: '7' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…n the ipfs-ecosystem group across 1 directory (#11405) * chore(deps): bump github.com/ipfs/go-ds-leveldb Bumps the ipfs-ecosystem group with 1 update in the / directory: [github.com/ipfs/go-ds-leveldb](https://github.com/ipfs/go-ds-leveldb). Updates `github.com/ipfs/go-ds-leveldb` from 0.5.2 to 0.5.3 - [Release notes](https://github.com/ipfs/go-ds-leveldb/releases) - [Commits](ipfs/go-ds-leveldb@v0.5.2...v0.5.3) --- updated-dependencies: - dependency-name: github.com/ipfs/go-ds-leveldb dependency-version: 0.5.3 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: ipfs-ecosystem ... Signed-off-by: dependabot[bot] <support@github.com> * chore: run make mod_tidy --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
upgrade to go-ipld-git v0.1.2
* fix: recover from panics in detached goroutines ls, dag get and dag export each hand their work to a goroutine and read the result back over a channel or pipe. Decoding and encoding there runs whatever codec a block's CID names, so it runs third-party code, and a panic on a detached goroutine ends the daemon rather than the command. Each now recovers, logs, and reports through the channel it already uses. In dag export the recover is registered after the existing cleanup defer so it runs first, while errCh is still open. * chore: update boxo, go-ipld-git and go-unixfsnode Picks up the CAR streaming and object parsing work from ipfs/boxo#1197, ipfs/go-ipld-git#77 and ipfs/go-unixfsnode#100. All three are pinned to their branches for now; swap for the tagged releases before merging. * chore: update boxo, go-ipld-git and go-unixfsnode go-ipld-git and go-unixfsnode are on their tagged releases. boxo is pinned to main, which carries ipfs/boxo#1197 but has not been released yet, so this still needs a boxo release before it can merge. Notes the CAR truncation marker in the v0.43 changelog, since that is the user-visible part of the boxo update. Also stops a slow Ubuntu mirror from failing the ipfs-webui job. That job installed Playwright OS dependencies for every browser although it declares no projects and so only ever runs chromium, and the install had no timeout of its own. When the mirror served 10.7 MB of package indices at 39 kB/s, apt-get update alone outlasted the job's 20 minute budget and the run was cancelled before any test started. The install is now scoped to chromium, capped, and best effort: the runner image already ships what headless chromium needs, and a library that really is missing surfaces when the browser fails to launch.
Bumps [actions/setup-go](https://github.com/actions/setup-go) from 6 to 7. - [Release notes](https://github.com/actions/setup-go/releases) - [Commits](actions/setup-go@v6...v7) --- updated-dependencies: - dependency-name: actions/setup-go dependency-version: '7' dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> (cherry picked from commit cae994a)
…n the ipfs-ecosystem group across 1 directory (#11405) * chore(deps): bump github.com/ipfs/go-ds-leveldb Bumps the ipfs-ecosystem group with 1 update in the / directory: [github.com/ipfs/go-ds-leveldb](https://github.com/ipfs/go-ds-leveldb). Updates `github.com/ipfs/go-ds-leveldb` from 0.5.2 to 0.5.3 - [Release notes](https://github.com/ipfs/go-ds-leveldb/releases) - [Commits](ipfs/go-ds-leveldb@v0.5.2...v0.5.3) --- updated-dependencies: - dependency-name: github.com/ipfs/go-ds-leveldb dependency-version: 0.5.3 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: ipfs-ecosystem ... Signed-off-by: dependabot[bot] <support@github.com> * chore: run make mod_tidy --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> (cherry picked from commit 867cdc5)
upgrade to go-ipld-git v0.1.2 (cherry picked from commit bb563a7)
* fix: recover from panics in detached goroutines ls, dag get and dag export each hand their work to a goroutine and read the result back over a channel or pipe. Decoding and encoding there runs whatever codec a block's CID names, so it runs third-party code, and a panic on a detached goroutine ends the daemon rather than the command. Each now recovers, logs, and reports through the channel it already uses. In dag export the recover is registered after the existing cleanup defer so it runs first, while errCh is still open. * chore: update boxo, go-ipld-git and go-unixfsnode Picks up the CAR streaming and object parsing work from ipfs/boxo#1197, ipfs/go-ipld-git#77 and ipfs/go-unixfsnode#100. All three are pinned to their branches for now; swap for the tagged releases before merging. * chore: update boxo, go-ipld-git and go-unixfsnode go-ipld-git and go-unixfsnode are on their tagged releases. boxo is pinned to main, which carries ipfs/boxo#1197 but has not been released yet, so this still needs a boxo release before it can merge. Notes the CAR truncation marker in the v0.43 changelog, since that is the user-visible part of the boxo update. Also stops a slow Ubuntu mirror from failing the ipfs-webui job. That job installed Playwright OS dependencies for every browser although it declares no projects and so only ever runs chromium, and the install had no timeout of its own. When the mirror served 10.7 MB of package indices at 39 kB/s, apt-get update alone outlasted the job's 20 minute budget and the run was cancelled before any test started. The install is now scoped to chromium, capped, and best effort: the runner image already ships what headless chromium needs, and a library that really is missing surfaces when the browser fails to launch. (cherry picked from commit f9baf8f)
* chore: upgrade to boxo v0.42.1 * use tagged release
* fix: update go-libp2p to patch webtransport CVE Picks up webtransport-go v0.11.1 and quic-go v0.60.0 through go-libp2p v0.49.0. webtransport-go v0.11.1 fixes CVE-2026-57497, unbounded buffering of unknown WebTransport capsules, which a peer could use to exhaust memory on any node with WebTransport enabled. Kubo enables it by default. v0.11.x also speaks WebTransport draft-15, so the server answers both the current browser handshake and the newer one Firefox is implementing. The tradeoff is that kubo can no longer dial peers running older go-libp2p over WebTransport. Those peers can still dial kubo, and go-to-go connections use /quic-v1 anyway. * docs: expand 0.43 changelog for go-libp2p 0.49 Reframe the webrtc-direct and WebTransport highlights around keeping browser retrieval working, and add the user-visible 0.49 fixes the changelog missed: - confirmed /webrtc-direct address dropped on the shared UDP port - Safari 26.4+ WebTransport handshake requirement - go-to-go WebTransport dial regression - relay backoff after losing a working relay, two shutdown races - quic-go HTTP/3 trailer CVE-2026-40898 in the security section * chore: bump boxo to v0.42.1 Moves the pin from a pre-release commit to the tag; the only commits between them are boxo's own go-libp2p v0.49.0 upgrade and release mechanics. Changelog: note the v0.42.1 fixes (IPv6 HTTP providers, shorter /routing/v1 stale windows) and the default CAR traversal depth cap of 1024 the gateway now inherits.
* chore: upgrade to boxo v0.42.1 * use tagged release (cherry picked from commit 9462845)
* fix: update go-libp2p to patch webtransport CVE Picks up webtransport-go v0.11.1 and quic-go v0.60.0 through go-libp2p v0.49.0. webtransport-go v0.11.1 fixes CVE-2026-57497, unbounded buffering of unknown WebTransport capsules, which a peer could use to exhaust memory on any node with WebTransport enabled. Kubo enables it by default. v0.11.x also speaks WebTransport draft-15, so the server answers both the current browser handshake and the newer one Firefox is implementing. The tradeoff is that kubo can no longer dial peers running older go-libp2p over WebTransport. Those peers can still dial kubo, and go-to-go connections use /quic-v1 anyway. * docs: expand 0.43 changelog for go-libp2p 0.49 Reframe the webrtc-direct and WebTransport highlights around keeping browser retrieval working, and add the user-visible 0.49 fixes the changelog missed: - confirmed /webrtc-direct address dropped on the shared UDP port - Safari 26.4+ WebTransport handshake requirement - go-to-go WebTransport dial regression - relay backoff after losing a working relay, two shutdown races - quic-go HTTP/3 trailer CVE-2026-40898 in the security section * chore: bump boxo to v0.42.1 Moves the pin from a pre-release commit to the tag; the only commits between them are boxo's own go-libp2p v0.49.0 upgrade and release mechanics. Changelog: note the v0.42.1 fixes (IPv6 HTTP providers, shorter /routing/v1 stale windows) and the default CAR traversal depth cap of 1024 the gateway now inherits. (cherry picked from commit 6020ab9)
* feat(cli): add --human and --sort-size to ipfs ls
- --human (-H): SI human-readable sizes in text output (humanize.Bytes)
- --sort-size (-S): sort directory entries by size, largest first
- Validation: --sort-size + --stream and --sort-size + --size=false errors
- Unit tests for formatSize and sort helpers
- CLI integration tests for both flags
- Changelog highlight in v0.44
* test(ls): make sort tests fail when sorting breaks
The tests covering --sort-size could not detect a broken feature. The
unit tests copied the comparator into the test body and sorted with their
own copy, so they passed regardless of what ls.go did. The CLI tests named
each fixture after its size, which left alphabetical order and size order
in agreement, so every ordering subtest passed even with --sort-size
disabled outright.
- extract lsLinkByName and lsLinkBySize so the tests exercise shipped code
- point the unit tests at those two functions
- name fixtures so their alphabetical order disagrees with their sizes
- pin the real directory behaviour: UnixFS directories carry no Filesize,
so they sort as 0, tie with empty files and break by name rather than
landing strictly last
* fix(ls): return 400 not 500 for bad flag combos
Passing --sort-size together with --stream or --size=false made
/api/v0/ls answer 500 Internal Server Error, telling API clients the
server had broken when the caller had simply combined flags that cannot
work together. Clients that retry on 5xx would retry a request that can
never succeed.
cmds.ErrClient maps to 400, matching how the rest of core/commands
reports caller mistakes. CLI output is unchanged.
* docs: fix --human size examples and JSON claims
The help for --human advertised "1K 234M 2G", which no kubo command has
ever printed. All three use humanize.Bytes, so the real output is SI with
a space: 1.2 kB, 234 MB, 2.0 GB. The stale example was copied into
'ipfs ls' from the two commands that already carried it, so correct all
three together.
- ls, repo stat, bitswap stat: examples now match real output
- drop the claim that --enc=json reports bytes. On the CLI, 'ipfs ls'
has a PostRun that prints the text table whatever --enc says, so no
JSON is produced there at all. Only the /api/v0/ls response is JSON,
and that part is true
- directories have no UnixFS Filesize, so they sort as 0 and tie with
empty files. They do not land strictly last, so stop saying they do
- changelog: a #### highlight with a TOC entry, kept short and pointing
at 'ipfs ls --help' for the details
- format sizes with strconv.FormatUint rather than fmt.Sprintf("%d")
---------
Co-authored-by: Marcin Rataj <lidel@lidel.org>
* chore: bump go-libp2p for sorted confirmed addrs Pin the head commit of libp2p/go-libp2p#3526: AutoNAT V2's ConfirmedAddrs returned unsorted buckets, and removeNotInSource silently dropped webrtc-direct from the confirmed set. Switch to a master pseudo-version once the PR merges. * fix: keep browser transports in provider records Provider records sent to HTTP routers were narrowed to the addresses AutoNAT V2 confirmed reachable, which silently dropped the only two transports a browser can dial: the AutoTLS /tls/ws address and webrtc-direct. A publicly reachable node was invisible to browser and Helia clients that found it through a delegated router, even though ipfs id and the DHT both advertised those addresses. AutoNAT only ever sees listen addresses, so the AutoTLS address, which the AddrsFactory synthesizes afterwards, can never reach the confirmed set. webrtc-direct does get confirmed, but go-libp2p loses it again in getConfirmedAddrs, which feeds an unsorted slice to a scan that assumes sorted input; that one is fixed upstream in libp2p/go-libp2p#3526. Announce host.Addrs() instead, the same set identify sends to peers and the DHT already publishes, narrowed to globally routable addresses so loopback and LAN entries stay out of a public index. Nodes with no public address keep announcing what they have, so LAN-only setups pointing at a local router are unaffected. - core/node/libp2p/routingopt.go: drop the ConfirmedAddrs branch from httpRouterAddrFunc, filter host.Addrs() with manet.IsPublicAddr; AppendAnnounce is emitted exactly once and does not count toward the public-addr check - core/commands/swarm_addrs_autonat.go: take over the BasicHost compile-time assertion, now the only ConfirmedAddrs consumer Fixes #11369 * docs: move highlight to v0.43 and scope it The fix ships in v0.43, so the entry moves out of v0.44.md and in next to the other browser-retrieval highlights. - names the config it applies to: Routing.Type=custom with a provide method on an HTTP router. Default auto provides over the DHT alone and is unaffected, since constructDefaultHTTPRouters leaves ProvideRouter as a noop. - cites bitsocial.net, which runs libp2p in the browser and uses delegated routers to find peers, as the app the gap broke
Telemetry reports to https://telemetry.ipshipyard.dev by default again, as it did through v0.42. This is a stopgap: it holds while the devgrant support window is active. Every way to turn telemetry off now lives in one place, so ending it later is a config change or a one-line diff rather than a rewrite. - endpoint is a linker-settable var: building with -ldflags "-X ...telemetry.defaultEndpoint=" yields a binary with no destination, which collects nothing and writes no identifier - DO_NOT_TRACK is honored, ranking between IPFS_TELEMETRY and the config Mode, so one variable opts a machine out of every tool - a collector answering 410 Gone retires itself: the node drops its identifier and never sends there again, on this run or a later one, which stops reporting across deployed nodes without a release - first-run notice names DO_NOT_TRACK next to the Kubo switches - docs/telemetry.md leads with how to disable, including at build time - AGENTS.md: telemetry opt-outs are a rule, not a courtesy - changelog: drop the opt-in highlight, v0.43 ships no telemetry change
* fix(routing): keep peers found before the timeout The DHT returns the closest peers it reached together with the context error when a lookup runs past its deadline. We dropped both, so any lookup slower than the routing server's per-request timeout came back as HTTP 500 with nothing in it, indistinguishable from a lookup that found no peers at all. Return what we have, and only error when the set is empty. * test: use local dht swarm for routing v1 test GetClosestPeers joined the public Amino DHT with real bootstrap peers, so the assertions depended on a CI runner reaching bootstrap.libp2p.io from a cold repo. When it could not, the test retried for five minutes and failed; ten such failures since v0.42.0, every one green on re-run. Bootstrap from the harness's in-process DHT peers instead, which the provider tests already use and this one predates. The window drops from five minutes to sixty seconds because there is no longer anything slow to wait for, and passing runs go from tens of seconds to under one. * test: stop handing out ports the kernel reuses NewRandPort binds port zero, notes the number, closes the socket and hands the number to the caller, which leaves a window for anything else on the machine to take it. The number also came from the ephemeral range, the same pool every outgoing connection draws from, and the CLI suite opens a lot of those. Both TestP2PForeground tunnel subtests died on "bind: address already in use" for a server the test binds itself. - NewTCPListener hands back the bound listener, closing that window for callers that listen in-process - ports for daemons we spawn now come from below the ephemeral range, so an outgoing connection cannot land on one * test: sync gc tests to the adder, not the clock TestAddGCLive asserted that gc had not started yet, but the only thing it waited for was the first file's output event. Between that event and the adder reaching the next file there is a gap, and the adder hands the pin lock to a waiting gc at exactly that boundary, so on a loaded runner gc really had started and the assertion was right to fail. Wrap the pipe so the test learns when the adder is inside the hanging file, and poll GCRequested instead of sleeping 100ms to know gc is queued. TestAddMultipleGCLive gets the same treatment for its two sleeps: too short there means gc never gets the lock and the test waits out its five second timeout instead. * test: move watched file in atomically os.WriteFile creates the file and fills it in two steps, and ipfswatch adds whatever is on disk when the create event wakes it. Catch it between the two and it adds an empty file, so the CID the test pulls out of the log reads back as nothing. Stage the file outside the watched directory and rename it in, which the watcher sees as one event for a file that is already complete. * test(sharness): poll the daemon request log The test backgrounded "ipfs log tail", slept 100ms and expected the daemon to be listing the request. The daemon only sees it once the client has started up and connected, which on a loaded runner takes longer than that, and then both the active and the inactive assertion fail together because the entry never appears at all. Poll for each state instead. The extra requests that polling makes push the daemon closer to the point where it drops finished entries from the log, so keep them with "diag cmds set-time" first. * test(sharness): drop stale peer count check The connect case opened by re-asserting that the previous case had left zero peers connected. Disconnecting is not permanent: the DHT keeps the other node in its routing table and re-dials it on any refresh, so that count is only true for as long as nothing else runs. What this case is named for, connecting with a bare /p2p/ address, is still covered by the connect itself and the peer count after it. * test(fuse): mount one node at a time Every parallel subtest does identical setup before mounting, so they all reach the mount together and around twenty setuid fusermount helpers open /dev/fuse inside the same instant. One occasionally comes back with a bare exit status 1. Take a lock for the mount call itself, which the subtests only hold for tens of milliseconds. Also report the failure instead of panicking: a panic failed all 37 tests in the package and left daemons behind, and the daemon's stderr, where fusermount says what actually went wrong, was captured and then thrown away. * test: compare cat output byte for byte The payload is 100 random bytes and the comparison ran through Trimmed(), which strips one trailing newline. Roughly one run in 256 ends in 0x0a and loses it. * test: wait for the fast-provide log line The daemon writes the line before it answers the RPC, but the test reads a buffer that a goroutine fills by copying the daemon's stderr, and that copy can still be behind when the command returns. Wait for the line rather than assuming it has landed. * test: allow for ipns republish mid-test A minute after the daemon starts, the republisher re-signs every key and publishes it again, giving the same value a new signature and expiry. The test captured one PUT body and compared it byte for byte with what routing returned, so a run slow enough to straddle that minute compared the first record against the second. Keep every record the mock is sent and require that routing's answer is one of them, which is what the assertion was reaching for. * fix(examples): turn off mdns in library example The example connects its two nodes by address, but left mDNS on, so local discovery could connect them first. A connection opened while a node is still being built is invisible to that node's bitswap, which only learns about connections made after it registers its notifier, and with no routing configured there is nothing to fall back on. The final fetch then waited forever and the test died on its two minute timeout with no clue why. Turning mDNS off makes the explicit dial the only way the two can meet, and keeps the example off the reader's LAN. Alongside that: - connectToPeers returns dial errors instead of logging and continuing into a fetch that cannot succeed - the example's own deadline now fits inside the test budget, so a stall names the step that hung - CommandContext so a hung child does not outlive the test * ci: make helia-interop job resilient Seven failures since v0.42.0 came from this job's setup rather than from any incompatibility. It installs whatever @helia/interop published last, and upstream shipped three packages in a row whose test config does not work from inside node_modules; a GitHub blip took out the rest. - find the compiled specs and pass them to aegir, instead of patching the config upstream ships into node_modules and grepping its text - pin node to a major: setup-node resolves an lts/ alias through a GitHub manifest with no retry and no fallback, and newer node rejects a flag aegir sets unconditionally - retry the registry lookup and fail loudly, since the old one-liner could not fail and left an empty cache key behind - install the exact version the cache key names, and only save the cache once the install is known good - drop the playwright apt packages, unused since this job stopped running browser targets
(cherry picked from commit c8ed996)
* chore: bump go-libp2p for sorted confirmed addrs Pin the head commit of libp2p/go-libp2p#3526: AutoNAT V2's ConfirmedAddrs returned unsorted buckets, and removeNotInSource silently dropped webrtc-direct from the confirmed set. Switch to a master pseudo-version once the PR merges. * fix: keep browser transports in provider records Provider records sent to HTTP routers were narrowed to the addresses AutoNAT V2 confirmed reachable, which silently dropped the only two transports a browser can dial: the AutoTLS /tls/ws address and webrtc-direct. A publicly reachable node was invisible to browser and Helia clients that found it through a delegated router, even though ipfs id and the DHT both advertised those addresses. AutoNAT only ever sees listen addresses, so the AutoTLS address, which the AddrsFactory synthesizes afterwards, can never reach the confirmed set. webrtc-direct does get confirmed, but go-libp2p loses it again in getConfirmedAddrs, which feeds an unsorted slice to a scan that assumes sorted input; that one is fixed upstream in libp2p/go-libp2p#3526. Announce host.Addrs() instead, the same set identify sends to peers and the DHT already publishes, narrowed to globally routable addresses so loopback and LAN entries stay out of a public index. Nodes with no public address keep announcing what they have, so LAN-only setups pointing at a local router are unaffected. - core/node/libp2p/routingopt.go: drop the ConfirmedAddrs branch from httpRouterAddrFunc, filter host.Addrs() with manet.IsPublicAddr; AppendAnnounce is emitted exactly once and does not count toward the public-addr check - core/commands/swarm_addrs_autonat.go: take over the BasicHost compile-time assertion, now the only ConfirmedAddrs consumer Fixes #11369 * docs: move highlight to v0.43 and scope it The fix ships in v0.43, so the entry moves out of v0.44.md and in next to the other browser-retrieval highlights. - names the config it applies to: Routing.Type=custom with a provide method on an HTTP router. Default auto provides over the DHT alone and is unaffected, since constructDefaultHTTPRouters leaves ProvideRouter as a noop. - cites bitsocial.net, which runs libp2p in the browser and uses delegated routers to find peers, as the app the gap broke (cherry picked from commit 4b7a94f)
Telemetry reports to https://telemetry.ipshipyard.dev by default again, as it did through v0.42. This is a stopgap: it holds while the devgrant support window is active. Every way to turn telemetry off now lives in one place, so ending it later is a config change or a one-line diff rather than a rewrite. - endpoint is a linker-settable var: building with -ldflags "-X ...telemetry.defaultEndpoint=" yields a binary with no destination, which collects nothing and writes no identifier - DO_NOT_TRACK is honored, ranking between IPFS_TELEMETRY and the config Mode, so one variable opts a machine out of every tool - a collector answering 410 Gone retires itself: the node drops its identifier and never sends there again, on this run or a later one, which stops reporting across deployed nodes without a release - first-run notice names DO_NOT_TRACK next to the Kubo switches - docs/telemetry.md leads with how to disable, including at build time - AGENTS.md: telemetry opt-outs are a rule, not a courtesy - changelog: drop the opt-in highlight, v0.43 ships no telemetry change (cherry picked from commit f64c770)
* fix(routing): keep peers found before the timeout The DHT returns the closest peers it reached together with the context error when a lookup runs past its deadline. We dropped both, so any lookup slower than the routing server's per-request timeout came back as HTTP 500 with nothing in it, indistinguishable from a lookup that found no peers at all. Return what we have, and only error when the set is empty. * test: use local dht swarm for routing v1 test GetClosestPeers joined the public Amino DHT with real bootstrap peers, so the assertions depended on a CI runner reaching bootstrap.libp2p.io from a cold repo. When it could not, the test retried for five minutes and failed; ten such failures since v0.42.0, every one green on re-run. Bootstrap from the harness's in-process DHT peers instead, which the provider tests already use and this one predates. The window drops from five minutes to sixty seconds because there is no longer anything slow to wait for, and passing runs go from tens of seconds to under one. * test: stop handing out ports the kernel reuses NewRandPort binds port zero, notes the number, closes the socket and hands the number to the caller, which leaves a window for anything else on the machine to take it. The number also came from the ephemeral range, the same pool every outgoing connection draws from, and the CLI suite opens a lot of those. Both TestP2PForeground tunnel subtests died on "bind: address already in use" for a server the test binds itself. - NewTCPListener hands back the bound listener, closing that window for callers that listen in-process - ports for daemons we spawn now come from below the ephemeral range, so an outgoing connection cannot land on one * test: sync gc tests to the adder, not the clock TestAddGCLive asserted that gc had not started yet, but the only thing it waited for was the first file's output event. Between that event and the adder reaching the next file there is a gap, and the adder hands the pin lock to a waiting gc at exactly that boundary, so on a loaded runner gc really had started and the assertion was right to fail. Wrap the pipe so the test learns when the adder is inside the hanging file, and poll GCRequested instead of sleeping 100ms to know gc is queued. TestAddMultipleGCLive gets the same treatment for its two sleeps: too short there means gc never gets the lock and the test waits out its five second timeout instead. * test: move watched file in atomically os.WriteFile creates the file and fills it in two steps, and ipfswatch adds whatever is on disk when the create event wakes it. Catch it between the two and it adds an empty file, so the CID the test pulls out of the log reads back as nothing. Stage the file outside the watched directory and rename it in, which the watcher sees as one event for a file that is already complete. * test(sharness): poll the daemon request log The test backgrounded "ipfs log tail", slept 100ms and expected the daemon to be listing the request. The daemon only sees it once the client has started up and connected, which on a loaded runner takes longer than that, and then both the active and the inactive assertion fail together because the entry never appears at all. Poll for each state instead. The extra requests that polling makes push the daemon closer to the point where it drops finished entries from the log, so keep them with "diag cmds set-time" first. * test(sharness): drop stale peer count check The connect case opened by re-asserting that the previous case had left zero peers connected. Disconnecting is not permanent: the DHT keeps the other node in its routing table and re-dials it on any refresh, so that count is only true for as long as nothing else runs. What this case is named for, connecting with a bare /p2p/ address, is still covered by the connect itself and the peer count after it. * test(fuse): mount one node at a time Every parallel subtest does identical setup before mounting, so they all reach the mount together and around twenty setuid fusermount helpers open /dev/fuse inside the same instant. One occasionally comes back with a bare exit status 1. Take a lock for the mount call itself, which the subtests only hold for tens of milliseconds. Also report the failure instead of panicking: a panic failed all 37 tests in the package and left daemons behind, and the daemon's stderr, where fusermount says what actually went wrong, was captured and then thrown away. * test: compare cat output byte for byte The payload is 100 random bytes and the comparison ran through Trimmed(), which strips one trailing newline. Roughly one run in 256 ends in 0x0a and loses it. * test: wait for the fast-provide log line The daemon writes the line before it answers the RPC, but the test reads a buffer that a goroutine fills by copying the daemon's stderr, and that copy can still be behind when the command returns. Wait for the line rather than assuming it has landed. * test: allow for ipns republish mid-test A minute after the daemon starts, the republisher re-signs every key and publishes it again, giving the same value a new signature and expiry. The test captured one PUT body and compared it byte for byte with what routing returned, so a run slow enough to straddle that minute compared the first record against the second. Keep every record the mock is sent and require that routing's answer is one of them, which is what the assertion was reaching for. * fix(examples): turn off mdns in library example The example connects its two nodes by address, but left mDNS on, so local discovery could connect them first. A connection opened while a node is still being built is invisible to that node's bitswap, which only learns about connections made after it registers its notifier, and with no routing configured there is nothing to fall back on. The final fetch then waited forever and the test died on its two minute timeout with no clue why. Turning mDNS off makes the explicit dial the only way the two can meet, and keeps the example off the reader's LAN. Alongside that: - connectToPeers returns dial errors instead of logging and continuing into a fetch that cannot succeed - the example's own deadline now fits inside the test budget, so a stall names the step that hung - CommandContext so a hung child does not outlive the test * ci: make helia-interop job resilient Seven failures since v0.42.0 came from this job's setup rather than from any incompatibility. It installs whatever @helia/interop published last, and upstream shipped three packages in a row whose test config does not work from inside node_modules; a GitHub blip took out the rest. - find the compiled specs and pass them to aegir, instead of patching the config upstream ships into node_modules and grepping its text - pin node to a major: setup-node resolves an lts/ alias through a GitHub manifest with no retry and no fallback, and newer node rejects a flag aegir sets unconditionally - retry the registry lookup and fail loudly, since the old one-liner could not fail and left an empty cache key behind - install the exact version the cache key names, and only save the cache once the install is known good - drop the playwright apt packages, unused since this job stopped running browser targets (cherry picked from commit 8ee3dcb)
Requested by @sanderpick in #11298 (comment)
Release v0.43.0
# Conflicts: # version.go
Merge release v0.43.0
Signed-off-by: weifanglab <weifanglab@outlook.com> Co-authored-by: Andrew Gillis <11790789+gammazero@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.