Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 9 additions & 2 deletions LIMITATIONS.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,8 +72,15 @@ do. Read this before trusting any single output.

## 6b. Verification limitations (experimental)

- Two built-in claims (billing-webhook-signature, jwt-authentication).
User-defined claims are not supported yet, deliberately.
- Four built-in claims (route-test-coverage, admin-authorization,
billing-webhook-signature, jwt-authentication). User-defined claims are not
supported yet, deliberately.
- Route test coverage is attributed statically (test imports and route names),
not by executing tests. A route exercised only indirectly can be reported as
uncovered.
- Admin authorization reports WEAK when it finds no authorization evidence.
That means DevTime found nothing, never that a route is confirmed
unprotected: global middleware and framework decorators are not detected.
- Verification is rule-driven over scanner signals; it inherits every scanner
coverage limitation listed here.
- Statuses mean "per DevTime's evidence rules", not formal proof or a security
Expand Down
4 changes: 2 additions & 2 deletions QUICKSTART.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ python -m venv .venv
source .venv/bin/activate # Windows (PowerShell): .venv\Scripts\Activate.ps1
# Windows (Git Bash): source .venv/Scripts/activate
pip install -e ".[dev]"
pytest # optional: all tests pass (129 at v0.4.0)
pytest # optional: all tests pass (142 at v0.5.0)
```

## 3. Create the demo repo
Expand Down Expand Up @@ -132,7 +132,7 @@ A fresh-clone check was run on the current candidate:
- **OS:** Windows 11 (Git Bash)
- **Python:** 3.11.9
- **Install:** `pip install -e ".[dev]"`
- **Tests:** all passing (129 at v0.4.0)
- **Tests:** all passing (142 at v0.5.0)
- **Demo:** `dtc init` / `dtc scan` / `dtc concepts` / `dtc explain "Billing Webhooks"`
all produced the expected output from a clean `git clone`.

Expand Down
11 changes: 7 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -165,7 +165,7 @@ Anything outside these six is out of scope for V0. See [LIMITATIONS.md](LIMITATI
| `dtc context <concept>` | Create a governed Context Pack for agents or humans. |
| `dtc risk --diff` | Review a git diff for risky changes using local evidence (advisory). |
| `dtc decision add` | Add a local decision record that can reduce uncertainty. |
| `dtc verify [claim]` | Verify a repository claim against evidence: status, contradictions, freshness (experimental). |
| `dtc verify [claim]` | Verify repository claims against evidence: status, contradictions, freshness (experimental). |

(Also available: `dtc evidence`, `dtc debt`, `dtc status`, `dtc doctor --privacy`,
`dtc export`, `dtc reset`, `dtc mcp start`.)
Expand Down Expand Up @@ -266,9 +266,12 @@ changes:

![dtc verify demo - a claim goes from SUPPORTED to CONTRADICTED to STALE](assets/devtime-verify-demo.svg)

Statuses are SUPPORTED, WEAK, CONTRADICTED, or UNKNOWN; contradictions always
show both sides; changed evidence marks a claim STALE. Two built-in claims ship
(billing webhook signatures, JWT authentication). See **[VERIFICATION.md](VERIFICATION.md)**.
Statuses are SUPPORTED, WEAK, CONTRADICTED, UNKNOWN, or NOT_APPLICABLE;
contradictions always show both sides; changed evidence marks a claim STALE.
Four built-in claims ship: route test coverage, admin authorization, billing
webhook signatures, and JWT authentication. `dtc verify` leads with what it can
actually verify here, and when nothing applies it says what would make a claim
verifiable instead of dead-ending. See **[VERIFICATION.md](VERIFICATION.md)**.

## Example output

Expand Down
105 changes: 105 additions & 0 deletions RELEASE_NOTES_v0.5.0.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
# DevTime v0.5.0 - claims that fire on ordinary repositories

The verification layer had a coverage problem. Its two claims (billing webhooks,
JWT) only apply to repositories that happen to have billing or JWT code. Run
against three realistic open-source repositories, `dtc verify` produced nothing
useful on two of them. Honest, but useless.

This release fixes that: claims that apply to ordinary repositories, plus an
output that never dead-ends. No cloud, no telemetry, no AI, no code execution -
unchanged.

## Two new built-in claims

**route-test-coverage** - "HTTP routes are exercised by tests."

```text
Route Test Coverage
Status: WEAK

Why:
- 6 of 15 routes have a referencing test.

Missing evidence:
- Tests referencing 9 route(s): /api/items/{id}, /api/login/access-token, ...
```

Routes and tests are the two most abundant kinds of evidence in almost every
server repository, so this claim fires nearly everywhere. Attribution is by test
imports and route names, and end-to-end specs are excluded because they match by
accident. Absence of tests is missing evidence, never a contradiction.

**admin-authorization** - "Administrative routes require an authorization check."

A missing authorization signal is reported as WEAK, never CONTRADICTED.
Authorization can be applied globally, by a router mount, or by a framework
decorator the scanner does not parse. Reporting an endpoint as unprotected when
it is not would destroy the trust this tool is built on, so DevTime says what it
found and what it cannot see.

## New status: NOT_APPLICABLE

A repository with no billing code is not "unknown" for a billing claim. The
claim simply does not apply. NOT_APPLICABLE says that plainly, and UNKNOWN is
now reserved for the harder case: the surface exists but the evidence cannot
decide.

## dtc verify is a report card

Findings first (contradictions before everything else), then a compact list of
claims that do not apply and why.

When nothing applies, DevTime no longer dead-ends. It reports what it scanned,
what evidence it collected, what would make a claim verifiable, and states
plainly that this is a coverage limit rather than a verdict on your code:

```text
No built-in claim applies to this repository yet.

DevTime scanned 129 files and found 199 signals.
Evidence collected: doc=199

Built-in claims become verifiable when a repository has:
- HTTP routes and tests (route-test-coverage)
...

This is a coverage limit, not a verdict on your repository.
```

`dtc verify --list` now shows which claims apply to the current repository.

## Fixed: MCP SDK 2.0 broke every fresh install

The MCP Python SDK released 2.0.0, which removed `mcp.server.fastmcp`. Any new
`pipx install "devtime-ei[mcp]"` resolved to the new SDK and could not start the
server at all. DevTime now supports both SDK generations (`MCPServer` in 2.x,
`FastMCP` in 1.x), verified against both, with a regression test so a future
rename cannot pass silently.

If you installed the MCP extra recently and `dtc mcp start` failed, this
release fixes it.

## Compatibility

- JSON output is `schema_version: 2`. Every version 1 field is unchanged; the
only addition is the NOT_APPLICABLE status value.
- Results that do not apply are no longer stored. Storing them would pollute
freshness and diff impact with claims that have no evidence.
- No command, concept, or MCP tool was renamed or removed.

## Notes

- 142 passing tests (20 new).
- Verified against three real repositories: an Express-based project went from
no findings to "59 of 133 routes have a referencing test"; a FastAPI template
went to "6 of 15 routes" plus SUPPORTED JWT authentication; a Go project
correctly reports that no claim applies and explains why.
- `dtc verify` remains sub-second on a repository with 133 routes and thousands
of tests.
- A secret-handling claim was investigated and deliberately not built: DevTime
hard-denies secret files from scanning, so evidence for that claim cannot
exist without breaking the trust model.

## Names

- PyPI distribution: `devtime-ei`. Python import: `devtime`. CLI: `dtc`.
38 changes: 32 additions & 6 deletions VERIFICATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,8 @@ guarantee.
| SUPPORTED | Required behavior evidence exists in the current scan. |
| WEAK | The claim's surface exists, but the proving evidence is missing. |
| CONTRADICTED | Credible evidence conflicts with the claim. Both sides are always shown. |
| UNKNOWN | No relevant surface was found, or coverage cannot responsibly decide. |
| UNKNOWN | The surface exists, but coverage cannot responsibly decide. |
| NOT_APPLICABLE | The repository has no surface this claim is about. |

## Freshness

Expand All @@ -55,8 +56,22 @@ same time: the last verification supported it, but its evidence changed since.
Freshness only tracks files that were evidence for the claim. Unrelated changes
never mark a claim stale.

NOT_APPLICABLE matters as much as the others. A repository with no billing code
is not "unknown" for a billing claim; the claim simply does not apply, and saying
so plainly is more useful than an ominous UNKNOWN.

## Built-in claims

- **route-test-coverage** (v0.5) - "HTTP routes are exercised by tests." Reports
how many routes have a referencing test and names the ones that do not.
Attribution is by test imports and route names; end-to-end specs are excluded
because they match by accident. Absence of tests is missing evidence, never a
contradiction.
- **admin-authorization** (v0.5) - "Administrative routes require an
authorization check." A missing authorization signal is WEAK, never
CONTRADICTED: authorization can be applied globally or by a wrapper the
scanner cannot see, and reporting an endpoint as unprotected when it is not
would destroy the trust this tool is built on.
- **billing-webhook-signature** - "Incoming billing webhooks verify the payment
provider's signature."
- **jwt-authentication** (v0.3) - "Authentication uses JWT access tokens."
Expand Down Expand Up @@ -85,6 +100,16 @@ Claim impact:
Only a claim's recorded evidence files count. A diff touching unrelated files
never flags a claim, and nothing is printed when no verified claim is affected.

## The report card (v0.5)

`dtc verify` leads with what it can actually say about your repository:
findings first (contradictions before everything else), then a compact list of
claims that do not apply and why.

When no claim applies, DevTime does not dead-end. It reports what it scanned,
what evidence it collected, what would make a claim verifiable, and states
plainly that this is a coverage limit rather than a verdict on your code.

## Trust model

- Deterministic and rule-driven. No AI, no network, no code execution.
Expand All @@ -100,14 +125,15 @@ never flags a claim, and nothing is printed when no verified claim is affected.
- Heuristic scanner: evidence comes from static patterns, not execution.
- Signature verification is recognized for known provider patterns
(e.g. Stripe `constructEvent`); custom schemes may not be detected.
- Two built-in claims. User-defined claims are deliberately not
- Four built-in claims. User-defined claims are deliberately not
supported yet: the claim model must earn trust before it grows a
configuration language.
- Coverage follows scanner language support; see [LIMITATIONS.md](LIMITATIONS.md).

## Where this is going

Next candidates, in order: more built-in claims over well-covered domains
(webhook idempotency), more contradiction detectors, and machine-readable
claim impact in risk output. User-defined
claims come after built-in claims prove trustworthy on real repositories.
Next candidates, in order: language coverage beyond the current TypeScript and
Python focus (Go repositories currently produce no routes at all), more built-in
claims over well-covered domains, and machine-readable claim impact in risk
output. User-defined claims come after built-in claims prove trustworthy on real
repositories.
4 changes: 2 additions & 2 deletions pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[project]
name = "devtime-ei"
version = "0.4.0"
version = "0.5.0"
description = "Local-first Engineering Intelligence for software repositories"
readme = "README.md"
requires-python = ">=3.11"
Expand Down Expand Up @@ -50,7 +50,7 @@ dev = [
Homepage = "https://github.com/Shakargy/devtime"
Repository = "https://github.com/Shakargy/devtime"
Issues = "https://github.com/Shakargy/devtime/issues"
"Release Notes" = "https://github.com/Shakargy/devtime/releases/tag/v0.4.0"
"Release Notes" = "https://github.com/Shakargy/devtime/releases/tag/v0.5.0"
Demo = "https://youtu.be/1Hiu3Y9J_SI"

[project.scripts]
Expand Down
4 changes: 2 additions & 2 deletions server.json
Original file line number Diff line number Diff line change
Expand Up @@ -8,12 +8,12 @@
"source": "github"
},
"websiteUrl": "https://github.com/Shakargy/devtime",
"version": "0.4.0",
"version": "0.5.0",
"packages": [
{
"registryType": "pypi",
"identifier": "devtime-ei",
"version": "0.4.0",
"version": "0.5.0",
"transport": {
"type": "stdio"
}
Expand Down
2 changes: 1 addition & 1 deletion src/devtime/__init__.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
"""DevTime - local-first Engineering Intelligence for repository memory."""

__version__ = "0.4.0"
__version__ = "0.5.0"

# Version metadata (Builder Edition, Chapter 20).
EVIDENCE_MODEL = "2026.06.1"
Expand Down
Loading
Loading