mozilla-history

Taskcluster history from https://firefox-ci-tc.services.mozilla.com deployment for:

• Hooks
• Roles
• Worker Pools

For https://community-tc.services.mozilla.com history, please see community-history.

Please note, until 9 November 2019 this repository stored the history for the https://taskcluster.net deployment. On 9 November 2019 the https://taskcluster.net deployment was decommissioned and the https://firefox-ci-tc.services.mozilla.com was instated.

Entity filenames

The files are named after the entities themselves, except for the following character conversions:

1. * -> ★

This conversion avoids illegal filenames.

2. / -> ⁄

Rather than creating nested subdirectories, this conversion avoids directory names colliding with entity filenames.

Installing

go get github.com/taskcluster/mozilla-history@v1.0.0

Running

unset TASKCLUSTER_CLIENT_ID TASKCLUSTER_ACCESS_TOKEN TASKCLUSTER_CERTIFICATE
export TASKCLUSTER_ROOT_URL='https://firefox-ci-tc.services.mozilla.com'
mozilla-history

This will populate subdirectories Hooks, Roles and WorkerPools of the current directory.

Refreshing All Local Data

For a complete local refresh, run:

./fetch_and_generate.py

When ~/.tc_token exists, the script loads its JSON clientId and accessToken fields. Otherwise, it uses TASKCLUSTER_CLIENT_ID and TASKCLUSTER_ACCESS_TOKEN from the environment. The token file takes precedence so stale exported credentials cannot silently override it.

The script builds temporary copies of the Go tools, refreshes Hooks, Roles, and WorkerPools, schedules worker-version probes, polls their sealed task group until every task reaches a terminal state, writes WorkerVersions, and stops. It does not pull, commit, push, or deploy anything. Review the generated changes before committing them.

The local refresh intentionally does not rebuild docs/history.json. History is derived exclusively from worker snapshots already committed to Git, so it cannot include the snapshot generated by the current run. The production publishing workflow rebuilds it after the snapshot is committed.

Use --poll-interval SECONDS to change the 60-second polling interval. If the script is interrupted after scheduling probes, resume without creating another group using --task-group-id TASK_GROUP_ID. TASKCLUSTER_ROOT_URL, REPORT_SCHEDULER_ID, and REPORT_PREFIX can be overridden in the environment.

Worker versions require two phases because Worker Manager configuration does not expose the implementation and version actually running in each pool. The first phase schedules an intentionally malformed task on every pool so the worker identifies itself in its task log. Once all tasks are resolved, the second phase reads those logs and generates the snapshot. Polling replaces the older fixed-delay assumption while preserving that probe-and-collect design.

Rendering an Existing Worker Snapshot

The worker-version report can be regenerated from a saved snapshot without Taskcluster credentials or probe tasks:

go run ./audit-worker-versions render \
  WorkerVersions/workers.json \
  WorkerVersions/README.md

Omit the output path to print the generated Markdown to standard output.

To preview the generated report in the website, serve the repository root:

./run_local.sh

Then open http://localhost:8000/docs/index-local.html. The local preview uses WorkerVersions/README.md and docs/history.json from the checkout. The published page continues to load the current report from GitHub. Pass a port as the first argument to override the default, for example ./run_local.sh 8080.

Testing

Pull requests and pushes to master run Go, Python, JavaScript, and Firefox layout tests through .github/workflows/ci.yml. These checks do not create Taskcluster probes or publish reports.

Optional Quick previews

See scripts/quick/README.md for personal Quick previews of the checkout's report. These helpers are separate from production publishing.

Production Automation

GitHub Actions (NAS replacement)

.github/workflows/reports.yml runs on Mondays at 07:23 UTC when enabled, or manually through Actions → Mozilla history reports → Run workflow. Install it on the default branch to enable dispatch and scheduling. Both production and forks skip scheduled report jobs unless the repository variable ENABLE_SCHEDULED_REPORTS is set to true. Add it under Settings → Secrets and variables → Actions → Variables → New repository variable to enable weekly runs. Set it to false or remove it to disable scheduled report jobs. Manual runs remain available regardless of this setting, including publishing when dry_run is disabled.

Configure repository Actions secrets TASKCLUSTER_CLIENT_ID and TASKCLUSTER_ACCESS_TOKEN with a dedicated Taskcluster client approved by the team. It needs permission to create lowest-priority probe tasks on the audited pools using scheduler smoketest, seal their task groups, and read the required configuration and artifacts. Have the Taskcluster owners scope this client to those operations; do not copy an unrestricted personal token into CI.

The job tests and builds the Go tools, snapshots configuration, creates the existing intentionally malformed probes, waits 90 minutes, and collects reports. It commits only Hooks, Roles, WorkerPools, WorkerVersions, and docs/history.json. History is rebuilt after committing the worker snapshot. An unchanged snapshot is successful. One report job runs at a time; a concurrent remote push causes publication to fail safely rather than overwrite changes. Publishing requires the default branch and branch rules permitting the Actions bot's normal push using GITHUB_TOKEN (contents: write).

Manual runs default to dry_run=true: they create real probes and generate a reviewable report-changes patch artifact, but do not push or deploy. Review the patch and task group in the run summary before running with dry_run disabled. Failures appear in Actions logs and run status; maintainers should enable Actions failure notifications. The job has a 150-minute timeout. If a run fails after creating probes, inspect its task group before retrying (a retry creates new probes). Keep the NAS job disabled during tests to avoid duplicate probes, and retire it after a successful publishing run in the team repository.

GitHub schedules run on the default branch and can be delayed; public-repository schedules are disabled after 60 days without repository activity. See GitHub schedule documentation.

GitHub Pages publishing

The production site is https://taskcluster.github.io/mozilla-history/. The repository's Settings → Pages → Build and deployment configuration is:

• Source: Deploy from a branch
• Branch: master
• Folder: /docs

The report workflow generates snapshots, commits them, rebuilds docs/history.json, and pushes the commits to master. GitHub's built-in Pages deployment publishes /docs from that branch. This path was verified with the October 14, 2026 report run: the report workflow's push was followed by a successful pages build and deployment run for the history commit. No separate repository Pages workflow or publishing variable is needed.

docs/index.html becomes the site's landing page; docs/migration.html is available as migration.html at the site root. The current worker report loads WorkerVersions/README.md and WorkerVersions/workers.json directly from the upstream repository, while historical charts load the deployed docs/history.json. A fresh current report alone does not confirm that Pages has deployed the latest history.

After publishing reports, check Actions → pages build and deployment for a successful deployment of the new history commit. Then check the live history.json for the new snapshot date.

Forks can configure the same branch-based Pages source to publish their docs/ directory, but the current worker report still loads upstream data. Use the local or Quick preview to view the checkout's own worker snapshot.

Legacy NAS scripts

The existing run-reports.sh and audit.sh scripts were used by the NAS job, which has been replaced by the report workflow above. They include git and production-site behavior; use fetch_and_generate.py for local refreshes.

Prerequisites

• Valid Taskcluster credentials must be set in the environment variables:
  • TASKCLUSTER_CLIENT_ID
  • TASKCLUSTER_ACCESS_TOKEN

How it works

1. run-reports.sh executes audit.sh
2. audit.sh schedules tasks for each worker pool to extract worker implementation details from logs
3. Results are stored in the WorkerVersions directory
4. mozilla-history stores Taskcluster configurations in their respective directories:
   • Hooks definitions in Hooks/
   • Roles definitions in Roles/
   • Worker Pool definitions in WorkerPools/
5. build-docs-history.sh collects every past revision of WorkerVersions/workers.json into docs/history.json, which feeds the two GitHub Pages views:
   • docs/index.html — the current report plus the full history tables and graphs
   • docs/migration.html — an animated month-by-month timeline of the docker-worker to generic-worker migration and of the generic-worker version rollout