Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,29 @@
Codebase overview
=================

| Area | Files | Role |
|------|-------|------|
| Core + DB | `__init__.py` | Regression model, `GitTree`/`GitBranch`, `ReportThread`, `run()`/`generate_web()`/`report()` |
| Bot commands | `_rbcmd.py` | Parse and execute `#regzbot` subcommands |
| CLI | `commandl.py` | argparse-based subcommands |
| Web export | `export_web.py` | Static HTML generation |
| Mail reports | `export_mail.py` | Weekly text/mail report layout |
| CSV export | `export_csv.py` | CSV-oriented export (tests) |
| Lore ingestion | `_repsources/_lore.py` | NNTP and HTTPS access to lore archives |
| Tracker sources | `_bugzilla.py`, `_gitlab.py`, `_github.py`, `_generic.py` | Tracker-specific API integrations |
| Tests | `testing_online.py`, `testing_offline.py`, `testing_trackers.py`, `testdata/*` | Offline/online/tracker tests/expected results |

Report sources (pluggable backends):

| Source | Implementation | Notes |
|--------|----------------|-------|
| **lore** (NNTP/HTTPS) | `_repsources/_lore.py` | Primary source; kernel mailing list archives |
| **bugzilla.kernel.org** | `_repsources/_bugzilla.py` | REST API with API key |
| **GitLab** | `_repsources/_gitlab.py` | Issue tracker integration |
| **GitHub** | `_repsources/_github.py` | Issue tracker integration |

Tracker polling logic lives in `_repsources/_trackers.py`.

Sign-off Process
================

Expand Down
82 changes: 67 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,24 +5,76 @@ driven Linux kernel development process. It's actually used in the field, but
still in a alpha stage: more adjustments are needed to make it better suite the
its intended purpose.

To get an impression how regression tracking with regzbot is performed in
practice, see the
[about page for the Linux kernel regression tracking efforts](https://linux-regtracking.leemhuis.info/about/)
and
[the list of regressions regzbot currently tracks](https://linux-regtracking.leemhuis.info/regzbot/mainline/).
If you want to interact with regzbot, check out
[getting started with regzbot](docs/getting_started.md) or the bots
[reference documentation](docs/reference.md).
Anyone can make regzbot track a regression by adding a command such as
`#regzbot introduced: <commit-id>` to an email sent to the Linux kernel mailing
list (LKML) or `regressions@lists.linux.dev`. Regzbot later notices fixes
automatically when a change points to the report using existing conventions such
as `Link:` or `Closes:` tags, which avoids extra work for developers. If those
links were forgotten, or if more details need to be recorded, the documentation
below lists other `#regzbot` commands.

To install or develop for regzbot, see the [installation documentation](docs/installation.md).
Kernel development treats regressions with high urgency — fixing them takes
priority over new features. Before automated tracking, regression reports on
mailing lists could disappear in the traffic. Regzbot automates the
bookkeeping: open regressions appear on a public dashboard, and periodic mail
summaries list outstanding issues ahead of releases. See the kernel
documentation for
[reporting regressions](https://docs.kernel.org/admin-guide/reporting-regressions.html) and
[handling regressions](https://docs.kernel.org/process/handling-regressions.html).

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I might be wrong, but isn't this at least in parts telling things a second time that are already covered?

Copy link
Copy Markdown
Author

@alanpeixinho alanpeixinho Jun 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, you are right. In the rearrangement some information became redundant. Gonna take a look.

## Licensing
## Scope

Rezbot is available under the LGPL 2.1; see the file COPYING for details.
Regzbot focuses on Linux kernel regressions in mainline, the latest mainline
release, and the stable series derived from it. Tracking is best-effort:
regzbot helps reports stay visible and helps connect fixes to reports, but it
does not guarantee every regression is tracked or fixed.

Regzbot was started by Thorsten Leemhuis as part of a project that has received
funding from the European Union’s Horizon 2020 research and innovation
It is not intended to track ordinary bugs, monitor every bug tracker used by
kernel subsystems, or produce complete regression statistics.

The public dashboards below show regression tracking in practice: start with
the mainline regression list to see currently tracked issues, while this README
gives background on the Linux kernel regression tracking effort. The
documentation links cover reporting, fixing, installation, and development.

### Public dashboards

| Resource | URL |
|----------|-----|
| Tracked regressions (mainline) | https://linux-regtracking.leemhuis.info/regzbot/mainline/ |
| All views (index) | https://linux-regtracking.leemhuis.info/regzbot/ |
| About the effort | [README](README.md) |
| Weekly reports (lore search) | [lore.kernel.org](https://lore.kernel.org/lkml/?q=%22Linux+regressions+report%22+f%3Aregzbot) |

### Documentation

* [Getting started](docs/getting_started.md) — report, fix, or update a tracked regression
* [Reference](docs/reference.md) — full `#regzbot` command syntax
* [Installation](docs/installation.md) — run your own instance
* [Contributing](CONTRIBUTING.md) — codebase layout and development

## History

Linux had no formal regression tracking for most of its history. Rafael J.
Wysocki tracked regressions for several years in the 2000s, but that effort
ended in 2012.

Thorsten Leemhuis restarted the work manually in 2017. That showed the value of
regression tracking, but also that manual tracking did not scale.

Development of regzbot started in 2021, and Thorsten began tracking
regressions with it later that year.

## Licensing and funding

Regzbot is available under the LGPL 2.1; see the file COPYING for details.

Regzbot was started by Thorsten Leemhuis as part of a project that received
funding from the European Union's Horizon 2020 research and innovation
programme under grant agreement No 871528.

Since May 2022 regzbot development and the Linux kernel regression tracking
efforts performed by Thorsten are supported with funds from Meta.
Starting in May 2022, regzbot development and the Linux kernel regression
tracking efforts performed by Thorsten were supported with funds from Meta for
about two years. Regzbot is now part of [KernelCI](https://kernelci.org/),
where it contributes to coordinated regression tracking across the kernel
ecosystem and receives funding for current development.
25 changes: 25 additions & 0 deletions docs/getting_started.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,19 @@
[[_TOC_]]


## Background

A **regression** is a change in the kernel that breaks something that previously
worked — degraded performance, a feature that stops working, or hardware that is
no longer recognized. Regressions are treated with higher urgency than ordinary
bugs; see the kernel documentation on
[reporting regressions](https://docs.kernel.org/admin-guide/reporting-regressions.html) and
[handling regressions](https://docs.kernel.org/process/handling-regressions.html).

This guide focuses on the `#regzbot` commands reporters and developers can use
to add or update tracked regressions.

Comment on lines +6 to +17
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This IMHO is not a good fit for the head of a "getting started" document, as it's a kinda boring text at the start of document (ask yourself: would you want to read a lengthy background sections like this when all you want is to use a software or understand what is it?). Something like this would be better.

""Regzbot is a bot tracking Linux kernel regressions over multiple domains (email, issue trackers, git trees, …) to ensure they are resolved in a timely manner – which is expected from Linux developers, as regressions (e.g. something that worked fine at one point works worse or not at all with a newer version) are forbidden.""

Needs a bit of fine tuning. One more sentence of that sort might be okay. But I guess all the other stuff in this section should go elsewhere or might be superfluous.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now I see that this is bringing some info from your personal https://linux-regtracking.leemhuis.info/about/ into the project.

I believe the "about" page of the repo is the README, so how do you feel about putting most of it in there?
Having the definition of regression in plain text is very helpful to understand the scope of the project

Copy link
Copy Markdown
Collaborator

@knurd knurd Jun 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now I see that this is bringing some info from […]

Ohh? Ohh, yeah, might totally be! 😀 To be honest I totally forgot about that page. 🫣

README, so how do you feel about putting most of it in there?

Sounds like a very good idea - at least unless it becomes a super long README. There might be overlap with getting_started, too. So one needs to be careful here.

@alanpeixinho is that something you would be willing to do as part of these changes or separately? Here is the source file for that page: about.md

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. I have included some of the info on README and geeting_started.
Do you think it is worth it keeping the historical information as well?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you think it is worth it keeping the historical information as well?

Not sure. Guess one or two sentences might be a worth it. Maybe even more or all of it; use your best judgement.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BTW, to avoid taking past each other: I still see the "Linux kernel development happens primarily over email. A few pieces of this infrastructure are relevant to regzbot: […] until it reaches the relevant tree." section in the patch (but maybe it's not the latest one?). I think that should also go or moved, as something like that feels misplaced and boring at the start of a "getting started" document.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I considered simplifying it a little bit, but I decided to just remove this part completely. As this information is already covered in other places.
I still kept the description of a regression. I think it is small enough to not bother that much, and might be useful.


## Why and how to make regzbot track a Linux kernel regression

When reporting a Linux kernel regression it is in your interest to make [regzbot](https://gitlab.com/knurd42/regzbot/) aware of the issue, as that ensures the report won't accidentally fall though the cracks; it also makes sure leading developers see the issue via the tracked regression website [or the weekly reports, which are not sent yet, but soon will be].
Expand All @@ -26,6 +39,18 @@ Regzbot is designed to normally not create any additional chores for Linux kerne
But sometimes you might want to do more with regzbot, like specifying a culprit exactly after a bisection or marking a regression as resolved. The text below explains how to do these and other things; the instructions there also will tell you how to use regzbot to track regressions for your own code or the subsystem you maintain, as that will make sure none fall through the cracks unnoticed.


## How to help keep tracking accurate

If a tracked issue turns out not to be a regression, if discussion moved
elsewhere, or if important details such as the title or introduced range are
wrong, please reply publicly and include the appropriate `#regzbot` command.
That keeps the public record clear and helps regzbot show useful status.

Developers do not need to CC individual regression trackers on every follow-up.
Keep `regressions@lists.linux.dev` in the loop and use `Link:` or `Closes:`
tags that point to the report when posting fixes.


## More regzbot features relevant for both reporters and developers


Expand Down
20 changes: 20 additions & 0 deletions docs/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,26 @@ Now you are ready to run regzbot

It will generate web reports at `~/.cache/regzbot/websites/`

## available commands

| Command | Purpose |
|---------|---------|
| `setup` | Initialize database, register sources, first Git tree sync |
| `run` | Full update cycle (sources → git → web) |
| `pages` | Regenerate web output only |
| `report` | Build interactive mail reports (operator sends manually) |
| `recheck` | Reprocess specific message IDs |
| `test` | Run offline/online test suites |

## data paths

| Path | Contents |
|------|----------|
| `~/.local/share/regzbot/database.db` | SQLite database (tracked regressions, processed message IDs, repository metadata) |
| `~/.cache/regzbot/gittrees/` | Local Git clones (mainline, next, stable) |
| `~/.cache/regzbot/websites/` | Generated static HTML output |
| `~/.config/regzbot/regzbot.cfg` | Configuration file (API keys) |

## Development tools

### Ruff
Expand Down