ci: report integration test results to Datadog Test Optimization#1270
ci: report integration test results to Datadog Test Optimization#1270jchrostek-dd wants to merge 3 commits into
Conversation
Instrument the GitLab integration-suite job's Jest run with dd-trace in agentless mode so per-test pass/fail/flake history is uploaded to Datadog. Lets us track the most-failing and flakiest integration tests over time. DD_API_KEY is already exported by get_secrets.sh and DD_SITE is set on the job; NODE_OPTIONS is set inline on the jest command only so it doesn't affect npm/tsc/cdk steps.
|
There was a problem hiding this comment.
Pull request overview
Note
Copilot couldn't run its full agentic review because no GitHub Actions runner was available. Make sure your repository has a runner available to run Copilot's review, or add a copilot-setup-steps.yml file specifying one with the runs-on attribute. See the docs for more details.
Adds Datadog CI Visibility / Test Optimization reporting for the integration test job so flaky/frequently failing tests can be tracked in Datadog.
Changes:
- Add
dd-traceto the integration-tests Node package. - Configure GitLab integration-suite job with Datadog CI Visibility env vars.
- Run Jest with
dd-trace/ci/initpreloaded to emit test telemetry.
Reviewed changes
Copilot reviewed 2 out of 3 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| integration-tests/package.json | Adds dd-trace dependency needed for Datadog CI Visibility instrumentation. |
| .gitlab/templates/pipeline.yaml.tpl | Sets Datadog CI Visibility configuration and preloads dd-trace/ci/init for Jest runs. |
Files not reviewed (1)
- integration-tests/package-lock.json: Generated file
| - echo "Running ${TEST_SUITE} integration tests with identifier ${IDENTIFIER}..." | ||
| - export TEST_SUITE=${TEST_SUITE} | ||
| - npx jest tests/${TEST_SUITE}.test.ts | ||
| - DD_TEST_SESSION_NAME="integration-${TEST_SUITE}" NODE_OPTIONS="-r dd-trace/ci/init" npx jest tests/${TEST_SUITE}.test.ts |
There was a problem hiding this comment.
I can only run on private repositories.
| "aws-cdk": "^2.1111.0", | ||
| "aws-cdk-lib": "^2.237.1", | ||
| "constructs": "^10.4.5", | ||
| "dd-trace": "^5.109.0", |
There was a problem hiding this comment.
I can only run on private repositories.
Overview
Adds Datadog's CI Health product to our integration tests. This will allow us to see what tests are failing frequently.
We are already able to see which test suites are failing, but this will include which specific test in the suite is failing.
Testing