Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .github/CODEOWNERS
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,10 @@ tools/ @DataDog/apm-common-components-core
windows/ @DataDog/libdatadog-core
fuzz/ @DataDog/chaos-platform

# Nix
*.nix @DataDog/nix-guild @DataDog/apm-common-components-core
Comment thread
lloeki marked this conversation as resolved.
flake.* @DataDog/nix-guild @DataDog/apm-common-components-core

# Specific overrides (must come after their general patterns above)
bin_tests/tests/test_the_tests.rs @DataDog/libdatadog-core
bin_tests/src/bin/test_the_tests.rs @DataDog/libdatadog-core
Expand All @@ -105,3 +109,4 @@ libdd-data-pipeline/tests/test_trace_exporter_otlp_export.rs @DataDog/apm-sdk-ca
libdd-trace-utils/src/otlp_encoder/ @DataDog/apm-sdk-capabilities-rust
datadog-sidecar/src/service/ffe_exposures_flusher.rs @DataDog/libdatadog-php @DataDog/libdatadog-apm @DataDog/feature-flagging-and-experimentation-sdk
datadog-sidecar/src/service/ffe_metrics_flusher.rs @DataDog/libdatadog-php @DataDog/libdatadog-apm @DataDog/feature-flagging-and-experimentation-sdk
.github/workflows/nix.yml @DataDog/nix-guild @DataDog/apm-common-components-core
91 changes: 91 additions & 0 deletions .github/workflows/nix.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
name: Test Nix
Comment thread
lloeki marked this conversation as resolved.

on: # yamllint disable-line rule:truthy
push:
branches:
- main
- mq-working-branch-*
Comment on lines +4 to +7

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
push:
branches:
- main
- mq-working-branch-*
# Nightly safeguard: catches devshell drift from changes that don't match the
# paths above (e.g. a dependency or build-script change that needs new native
# tooling). Scheduled runs only fire from the default branch, against its HEAD.
schedule:
- cron: "0 4 * * *" # daily at 04:00 UTC

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of triggering it in every PR I think it would be better to have a nightly schedule. That way we can save some capacity for the actual pipelines.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But wouldn't you want to check tings on PRs that change Nix files at least?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Definitely.

As is it's not going to trigger in every PR anyway, only those that change nix files or that have an impact on nix (the toolchain files).

The schedule can be nice though as a sanity check for when those don't change often.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added schedule

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But as I undestand the workflow, the job will be executed whenever any of the files involved is modified as in:

pull_request:
    paths:
      - "*.nix"
      - "flake.*"
      - "rust-toolchain.toml"
      - ".github/workflows/nix.yml"  

Then it will be executed nightly as a safe guard so I think there's no need to trigger it in every PR on the mq/main.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes for pull requests, but not for pushes to main (e.g merge commit) or merge queue (so it will not fire and maybe fail a possible required check constraint; I'm not sure of MQ behaviour here)

If that's acceptable then it can be removed.

# Also run on PRs that touch the devshell or what it reads. Paths mirror the
# Nix CODEOWNERS entries, plus the toolchain files (read by the flake) and
# this workflow itself.
pull_request:
paths:
- "*.nix"
- "flake.*"
- "rust-toolchain.toml"
- "nightly-toolchain.toml"
- ".github/workflows/nix.yml"
# Nightly safeguard: catches devshell drift from changes that don't match the
# paths above (e.g. a dependency or build-script change that needs new native
# tooling). Scheduled runs only fire from the default branch, against its HEAD.
schedule:
- cron: "0 4 * * *" # daily at 04:00 UTC

# Default permissions for all jobs
permissions: {}

concurrency:
group: ci-${{ github.ref == 'refs/heads/main' && github.run_id || github.ref }}-nix
cancel-in-progress: true

env:
CARGO_TERM_COLOR: always

jobs:
test:
strategy:
fail-fast: false
matrix:
platform:
- os: darwin
cpu: arm64
base: macos-15 # always arm64-darwin
Comment thread
lloeki marked this conversation as resolved.
- os: linux
cpu: x86_64
base: ubuntu-24.04 # always x86_64-linux-gnu
- os: linux
cpu: aarch64
base: ubuntu-24.04-arm # always aarch64-linux-gnu
Comment thread
lloeki marked this conversation as resolved.

name: Test Nix (${{ matrix.platform.cpu }}-${{ matrix.platform.os }})
runs-on: ${{ matrix.platform.base }}

permissions:
contents: read

steps:
- name: Check CPU arch
run: |
test "$(uname -m)" = "${{ matrix.platform.cpu }}"
- name: Free Disk Space (Linux only)
if: runner.os == 'Linux'
uses: jlumbroso/free-disk-space@54081f138730dfa15788a46383842cd2f914a1be # 1.3.1
with:
tool-cache: true
android: true
dotnet: true
haskell: true
large-packages: false
docker-images: false
swap-storage: false
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
persist-credentials: false
- uses: cachix/install-nix-action@ab739621df7a23f52766f9ccc97f38da6b7af14f # v31.10.5
- name: Print toolchain versions
run: |
nix develop --command rustc --version
nix develop --command cargo --version
nix develop --command cbindgen --version
- name: Check nightly formatter toolchain
run: nix develop .#nightly --command cargo fmt --version
- name: Build workspace
run: nix develop --command cargo build --workspace --exclude builder

complete:
name: Nix (complete)
runs-on: ubuntu-24.04
needs:
- test
steps:
- run: echo "DONE!"
46 changes: 46 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,52 @@ We provide two Dev Container configurations:

The container includes all necessary dependencies for building and testing `libdatadog`.

#### Nix development shell

The Nix flake provides a reproducible, pinned shell with Rust, `cbindgen`, and native build tools.

This works natively under both Linux and Darwin.

##### Prerequisite: install Nix

This one-liner installs Nix isolated in `/nix`:

```bash
curl --proto '=https' --tlsv1.2 --location https://nixos.org/nix/install | sh -s -- --daemon
```

Enable the modern CLI and flakes in `/etc/nix/nix.conf`:

```bash
echo "experimental-features = nix-command flakes" | sudo tee -a /etc/nix/nix.conf
```

See the [Nix manual](https://nix.dev/manual/nix/2.28/installation/index.html) for more information.

##### Spawn an interactive shell

Spawn a shell with an environment set up to expose the tooling:

```bash
nix develop
```

Note: legacy `nix-shell` and `nix-build` are also available via the `flake-compat` shims.

##### Run commands

Alternatively, run individual commands:

```bash
nix develop --command cargo build --workspace --exclude builder
nix develop .#nightly --command cargo fmt --all -- --check
```

##### Debugging CI failures

- Reproduce the Nix CI build with `nix develop --command cargo build --workspace --exclude builder`.
- After an MSRV or nightly bump, update `rust-toolchain.toml` or `nightly-toolchain.toml`, then refresh `flake.lock` with `nix flake update`.

#### Docker container
A dockerfile is provided to run tests in a Ubuntu linux environment. This is particularly useful for running and debugging linux-only tests on macOS.

Expand Down
14 changes: 14 additions & 0 deletions default.nix
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# Copyright 2026-Present Datadog, Inc. https://www.datadoghq.com/
# SPDX-License-Identifier: Apache-2.0

# flake-compat shim for usage without flakes
(import
(
let lock = builtins.fromJSON (builtins.readFile ./flake.lock); in
fetchTarball {
url = lock.nodes.flake-compat.locked.url or "https://github.com/edolstra/flake-compat/archive/${lock.nodes.flake-compat.locked.rev}.tar.gz";
sha256 = lock.nodes.flake-compat.locked.narHash;
}
)
{ src = ./.; }
).defaultNix
97 changes: 97 additions & 0 deletions flake.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

62 changes: 62 additions & 0 deletions flake.nix
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
# Copyright 2026-Present Datadog, Inc. https://www.datadoghq.com/
# SPDX-License-Identifier: Apache-2.0

{
Comment thread
lloeki marked this conversation as resolved.
inputs = {
nixpkgs.url = "github:nixos/nixpkgs/release-26.05";

# cross-platform convenience
flake-utils.url = "github:numtide/flake-utils";

# backwards compatibility with nix-build and nix-shell
flake-compat.url = "https://flakehub.com/f/edolstra/flake-compat/1.tar.gz";
Comment thread
lloeki marked this conversation as resolved.

# pinned, exact upstream Rust toolchains
rust-overlay = {
url = "github:oxalica/rust-overlay";
inputs.nixpkgs.follows = "nixpkgs";
};
};

outputs = { self, nixpkgs, flake-utils, flake-compat, rust-overlay }:
# resolve for all platforms in turn
flake-utils.lib.eachDefaultSystem (system:
let
# packages for this system platform, with the rust-overlay applied
pkgs = import nixpkgs {
inherit system;
overlays = [ (import rust-overlay) ];
};

# A devshell for a given Rust toolchain (read from a toolchain file via
# rust-overlay), with the rest of the build dependencies.
mkDevShell = rust: pkgs.mkShell {
name = "libdatadog-devshell";

# The stdenv cc-wrapper injects -D_FORTIFY_SOURCE, which glibc rejects
# when compiling without optimization. Some build scripts (e.g.
# spawn_worker's trampoline.c) compile C at -O0 with -Werror, so the
# resulting fortify #warning becomes a hard error. Disable fortify
# hardening in the shell so those builds succeed.
hardeningDisable = [ "fortify" "fortify3" ];

nativeBuildInputs = [
rust # rustc + cargo + rustfmt + clippy, pinned via toolchain file
Comment thread
lloeki marked this conversation as resolved.
pkgs.rust-cbindgen
Comment thread
lloeki marked this conversation as resolved.
pkgs.cmake
pkgs.autoconf
pkgs.automake
pkgs.libtool
];
};
in {
# Default: the pinned stable toolchain (single source of truth is
# ./rust-toolchain.toml), matching CI and rustup.
devShells.default = mkDevShell (pkgs.rust-bin.fromRustupToolchainFile ./rust-toolchain.toml);

# Nightly toolchain (./nightly-toolchain.toml) for the jobs that
# genuinely need a nightly compiler. Use with `nix develop .#nightly`.
devShells.nightly = mkDevShell (pkgs.rust-bin.fromRustupToolchainFile ./nightly-toolchain.toml);
}
);
}
14 changes: 14 additions & 0 deletions shell.nix
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# Copyright 2026-Present Datadog, Inc. https://www.datadoghq.com/
# SPDX-License-Identifier: Apache-2.0

# flake-compat shim for usage without flakes
(import
(
let lock = builtins.fromJSON (builtins.readFile ./flake.lock); in
fetchTarball {
url = lock.nodes.flake-compat.locked.url or "https://github.com/edolstra/flake-compat/archive/${lock.nodes.flake-compat.locked.rev}.tar.gz";
sha256 = lock.nodes.flake-compat.locked.narHash;
}
)
{ src = ./.; }
).shellNix
Loading