chore: add support to build libdatadog from a commit reference

[hoolioh]

Why?

rake libdatadog:build could only build the single pinned v<LIB_VERSION> tag from
libdatadog's GitHub repo, plus a local checkout via LIBDATADOG_SOURCE. Support for a
specific tag/commit was then added as separate LIBDATADOG_TAG / LIBDATADOG_COMMIT
variables — one variable per source type, which is verbose, has no option for building a
branch, and needs mutual-exclusion validation to police the multi-var design.

This consolidates all of them into a single LIBDATADOG_REF variable and adds first-class
branch support.

What does this PR do?

Replaces LIBDATADOG_SOURCE / LIBDATADOG_TAG / LIBDATADOG_COMMIT with one variable:

• LIBDATADOG_REF — an explicit <kind>:<value> pair, where kind is one of:
  • tag:v33.0.0 → cargo install --git … --tag
  • branch:my-feature → cargo install --git … --branch (new — branches are now first-class)
  • commit:<sha> → cargo install --git … --rev
  • path:/path/to/checkout → cargo install --path

The kind is always stated explicitly — there is no auto-detection — so the source type is
never guessed and a value without a recognized prefix fails fast with a clear message.
Behavior is unchanged when LIBDATADOG_REF is unset: it builds the pinned v<LIB_VERSION>
tag.

Git builds (tag/branch/commit) pass cargo install --locked, so the builder is built
reproducibly against the dependency versions pinned in libdatadog's Cargo.lock
(supply-chain hardening). Local-path builds intentionally omit --locked, since the
checkout may be modified locally.

The old LIBDATADOG_SOURCE / LIBDATADOG_TAG / LIBDATADOG_COMMIT variables are removed
(hard cut — LIBDATADOG_COMMIT was never released). To avoid silently ignoring stale
usage, the task raises a clear error pointing to LIBDATADOG_REF if any of them is set.

Artifacts from an explicit-ref build now land in a directory named after the ref value
(vendor/libdatadog-v32.0.0/, vendor/libdatadog-main/, vendor/libdatadog-<sha>/,
vendor/libdatadog-<checkout-basename>/) instead of always vendor/libdatadog-<LIB_VERSION>/,
so a non-release build is no longer mislabeled as the pinned version. The default build is
unchanged and still uses the canonical libdatadog-<LIB_VERSION> location that packaging and
the runtime look up. rake libdatadog:clean now removes every vendor/libdatadog-* tree.

Implementation

BuildFromSource::Builder.parse_ref splits an explicit <kind>:<value> string into
[kind, value] (or raises on an unrecognized/missing prefix).
BuildFromSource::Builder.cargo_install_cmd(ref:, features:) then routes each kind to its
canonical cargo flag via the GIT_FLAG map, special-casing path: (uses --path, no
--locked), and defaults to the pinned --tag v<LIB_VERSION> when ref is blank.

The libdatadog:build task reads LIBDATADOG_REF / LIBDATADOG_FEATURES from the
environment, guards against the removed legacy vars, passes the ref straight through, and
logs which ref it is building.

How to test the change?

Build a specific tag (requires a Rust toolchain):

LIBDATADOG_REF=tag:v33.0.0 bundle exec rake libdatadog:build

Build a branch or a specific commit:

LIBDATADOG_REF=branch:main bundle exec rake libdatadog:build
LIBDATADOG_REF=commit:<sha> bundle exec rake libdatadog:build

[lloeki]

Why would that be?

I think it would be entirely reasonable for tag/ref to be usable with a local source (git clone local/path somewhere/else absolutely works)

With the presumed tag/ref argument unification noted nearby this whole check would go away.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b2d7db89d3

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[chatgpt-codex-connector]

Treat empty LIBDATADOG_REF as default

When a CI job or shell wrapper exports LIBDATADOG_REF as an empty string, ENV["LIBDATADOG_REF"] is truthy in Ruby, so this calls parse_ref("") and raises instead of building the pinned tag. The task below already treats ref && !ref.empty? as the explicit-ref case, so this makes default builds fail only when the optional env var is present but blank; use the same blank check here before parsing.

Useful? React with 👍 / 👎.

[ivoanjo]

So I've tried this with LIBDATADOG_REF=branch:ivoanjo/prof-150465-experiment-omit-root-span-id bundle exec rake libdatadog:build and I get a build out of it. Is there a command for me to get some .gem files out of it in one-shot?