Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions examples/skills_code_review_agent/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
__pycache__/
*.pyc
output/
*.db
.env
23 changes: 23 additions & 0 deletions examples/skills_code_review_agent/DESIGN.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# Design: Automated Code Review Agent

## Architecture

The agent follows a pipeline architecture with five stages:

```
Input(diff) -> Parser -> Rules -> Dedup+Redact -> Storage+Report
```

**Skill Design**: The code-review skill provides 4 rule categories (security, resource leaks, error handling, testing) and two sandbox scripts for independent execution. The ten deterministic regex rules guarantee baseline detection rates without LLM dependency, enabling dry-run mode that completes in under one second. Rules are confidence-weighted: critical findings default to 0.95 confidence while testing rules at 0.80 route to human-review warnings.

**Sandbox Isolation**: Script execution uses local subprocess executor with timeout control (30s default) and output size limits (100KB). Container executor interface is reserved for production deployment where network isolation and read-only mounts are required. Execution failures are captured as partial results without aborting the review pipeline.

**Filter Strategy**: Three-level pre-execution governance classifies commands into deny (system destruction like rm -rf, mkfs, fork bombs — blocked from sandbox entirely), ask (privilege escalation like sudo, chown — requires user confirmation), and needs_human_review (dependency installs like pip install, network calls like curl — flagged for human judgment). Denied findings are recorded in filter_decision table and excluded from the report. Review-flagged items remain present but with filter_action metadata for auditor visibility.

**Filter enforcement (acceptance criterion #7)**: criterion #7 says `deny / needs_human_review` must not directly reach sandbox execution. This example enforces it literally: `deny` is always hard-blocked, while `ask` and `needs_human_review` share the same pre-execution decision gate — an interactive operator confirmation (`y/N`, or auto-reject with `--non-interactive`). Nothing executes without an operator decision, and every decision (approved/denied) is recorded in filter_decision rows and the report "Filter Intercepts" section. In Agent mode, the filter is attached to `skill_run` via `SkillToolSet(run_tool_kwargs={"filters": [CodeReviewSafetyFilter(...)]})` so enforcement runs before execution; both `confirm`/`record` callbacks are injectable for tests.

**Database Schema**: Six SQLite tables connected by task_id foreign keys form a complete audit trail: review_task for job metadata, finding for structured issues, sandbox_run for execution records, filter_decision for governance decisions, monitoring for performance metrics with JSON severity distribution, and report for generated outputs. The get_task_details method aggregates all records for a single task.

**Dedup and Noise Reduction**: Findings are deduplicated using (file, line, category) triplets as composite keys. Items with confidence below 0.85 are routed to warnings for human review rather than mixed with high-confidence findings. Testing violations and other inherently uncertain categories automatically fall into the warning bucket.

**Security Boundaries**: Sensitive data including API keys, access tokens, bearer credentials, JWT tokens, and passwords is redacted at multiple boundaries: finding evidence before database writes, sandbox stdout/stderr before persistence, and report content before output. Redaction combines seven regex pattern families with Shannon entropy detection for identifying unknown high-entropy credential strings.
81 changes: 81 additions & 0 deletions examples/skills_code_review_agent/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
# Skills Code Review Agent

Automated code review agent for Python projects using Skills, deterministic rules, sandbox execution, and SQLite persistence.

## Quick Start

```bash
cd examples/skills_code_review_agent

# Sync mode: review a diff file or git repo
python run_review.py --diff-file fixtures/security.diff
python run_review.py --repo-path /path/to/your/repo

# Agent mode: LlmAgent + SkillToolSet + FakeModel (no API key)
python run_review.py --diff-file fixtures/security.diff --agent

# Agent mode with real LLM enhancement
export TRPC_AGENT_API_KEY=your_key
export TRPC_AGENT_BASE_URL=https://api.openai.com/v1
export TRPC_AGENT_MODEL_NAME=gpt-4o-mini
python run_review.py --diff-file fixtures/security.diff --agent --model gpt-4o-mini

# Run all tests
python -m pytest test_code_review_agent.py -v
```

## Modes

| Mode | Flag | Driver | Skill Loading | Sandbox | Filter |
|------|------|--------|---------------|---------|--------|
| Sync (default) | `--diff-file` | `main()` function pipeline | not used | `SandboxRunner.run_script()` | `check_dangerous()` |
| Agent | `--agent` | `LlmAgent` + `Runner.run_async()` | `skill_load`/`skill_run` | `SkillRunTool` + `LocalWorkspaceRuntime` | `CodeReviewSafetyFilter(BaseFilter)` |

## Filter Governance (three-level semantics)

Commands found in tool calls / finding evidence are classified into three levels:

| Level | Examples | Behavior |
|-------|----------|----------|
| `deny` | `rm -rf /`, `mkfs`, fork bomb | Hard-blocked (`is_continue=False`), never executed, decision recorded |
| `ask` | `sudo`, `iptables`, `chown` | Interactive confirmation prompt (`y/N`); approved → executes and recorded as `approved`, rejected → blocked and recorded as `denied (no human confirmation)`. Use `--non-interactive` to auto-reject without prompting |
| `needs_human_review` | `pip install`, `curl`, `wget` | **Same decision gate as `ask`**: interactive confirmation before execution; approved → executes and recorded as `approved`, rejected → blocked and recorded as `denied (no human confirmation)`. Use `--non-interactive` to auto-reject |

**Acceptance criterion #7 compliance:** criterion #7 states that `deny / needs_human_review` must not directly reach sandbox execution. This example enforces that literally: neither `deny` nor `needs_human_review` commands execute without an operator decision. `deny` is always blocked; `ask`/`needs_human_review` prompt interactively (or auto-reject with `--non-interactive`), and every decision (approved/denied) is recorded in the filter_decision table and surfaced in the report "Filter Intercepts" section. In Agent mode the filter is attached to `skill_run` via `SkillToolSet(run_tool_kwargs={"filters": [CodeReviewSafetyFilter(...)]})`, so enforcement happens before execution.

## Review Categories

| Category | Severity | Examples |
|----------|----------|----------|
| Security | critical | Hardcoded secrets, shell=True, eval(), pickle.loads |
| Resource Leak | high | open() without with, unclosed HTTP sessions, DB connections |
| Error Handling | high | Swallowed exceptions, bare except clauses |
| Testing | medium | New functions/classes without test coverage |

## Test Fixtures

| Fixture | Description | Expected |
|---------|-------------|----------|
| clean.diff | Trivial helper function | 0 findings |
| security.diff | Hardcoded secrets, eval, pickle, shell=True | >=3 security findings |
| resource_leak.diff | open() without with, unclosed DB connection | >=2 resource findings |
| db_lifecycle.diff | pymysql.connect without context manager | >=1 DB lifecycle finding |
| missing_test.diff | New functions without test file | >=1 testing warning |
| duplicate.diff | Multiple issues on same file | Proper dedup |
| sandbox_fail.diff | Long-running script | No crash, partial results |
| sensitive_info.diff | API keys, tokens, passwords | All secrets redacted |

## Output

- `output/<task_id>/review_report.json` — JSON structured report
- `output/<task_id>/review_report.md` — Markdown human-readable report

## Architecture

See [DESIGN.md](DESIGN.md) for architecture details.

## Requirements

- Python 3.10+
- trpc-agent-py >= 1.1.0
- pytest (for tests only)
Empty file.
37 changes: 37 additions & 0 deletions examples/skills_code_review_agent/agent/dedup.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
# Tencent is pleased to support the open source community by making trpc-agent-python available.
# Copyright (C) 2025 Tencent. All rights reserved.
# trpc-agent-python is licensed under the Apache License Version 2.0.
"""Finding deduplication and confidence-based routing."""

from typing import Any


def dedup_findings(findings: list[dict[str, Any]]) -> tuple[list[dict], list[dict]]:
"""Deduplicate findings by (file, line, category).

Low-confidence findings (< 0.85) are routed to warnings.
High-confidence findings are deduplicated.

Each pool (findings, warnings) uses its own dedup set so a
low-confidence hit never shadows a high-confidence one and vice versa.

Returns (findings, warnings) tuple.
"""
seen_high: set[tuple] = set()
seen_warn: set[tuple] = set()
high_conf: list[dict] = []
warnings: list[dict] = []

for f in findings:
key = (f.get('file', ''), f.get('line', 0), f.get('category', ''))

if f.get('confidence', 1.0) < 0.85:
if key not in seen_warn:
seen_warn.add(key)
warnings.append(f)
else:
if key not in seen_high:
seen_high.add(key)
high_conf.append(f)

return high_conf, warnings
93 changes: 93 additions & 0 deletions examples/skills_code_review_agent/agent/diff_parser.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
# Tencent is pleased to support the open source community by making trpc-agent-python available.
# Copyright (C) 2025 Tencent. All rights reserved.
# trpc-agent-python is licensed under the Apache License Version 2.0.
"""Unified diff parser — converts git diff text into structured data."""

import re
from typing import Any


HUNK_HEADER_RE = re.compile(r'^@@ -(\d+)(?:,(\d+))? \+(\d+)(?:,(\d+))? @@')
ONLY_FILES_RE = re.compile(r'^diff --git a/(.*) b/(.*)$')
NEW_FILE_RE = re.compile(r'^new file mode')
DELETED_FILE_RE = re.compile(r'^deleted file mode')


def parse_diff(diff_text: str) -> dict[str, Any]:
"""Parse unified diff into structured data.

Returns dict with:
- files: list of file change objects
- total_added_lines: int
- _all_hunks: flat list of all hunks
"""
files: list[dict[str, Any]] = []
all_hunks: list[dict[str, Any]] = []
current_file: dict[str, Any] | None = None
current_hunk: dict[str, Any] | None = None

for line in diff_text.splitlines():
if line.startswith('diff --git '):
m = ONLY_FILES_RE.match(line)
if m:
current_file = {
'path': m.group(2),
'old_path': m.group(1),
'hunks': []
}
files.append(current_file)
current_hunk = None
continue

if not current_file:
continue

if line.startswith('--- ') or line.startswith('+++ '):
continue

hunk_match = HUNK_HEADER_RE.match(line)
if hunk_match:
current_hunk = {
'old_start': int(hunk_match.group(1)),
'old_count': int(hunk_match.group(2) or 1),
'new_start': int(hunk_match.group(3)),
'new_count': int(hunk_match.group(4) or 1),
'added_lines': [],
'context_before': [],
'file_path': current_file['path'],
'_line_counter': int(hunk_match.group(3)),
}
current_file['hunks'].append(current_hunk)
all_hunks.append(current_hunk)
continue

if current_hunk is None:
continue

if line.startswith('+') and not line.startswith('+++'):
text = line[1:] if len(line) > 1 else ''
current_hunk['added_lines'].append({
'line': current_hunk['_line_counter'],
'text': text,
'context': list(current_hunk['context_before'][-3:])
})
current_hunk['_line_counter'] += 1
elif not line.startswith('-'):
ctx_text = line[1:] if line.startswith(' ') else line
current_hunk['context_before'].append(ctx_text)
current_hunk['_line_counter'] += 1

total_added = sum(
sum(len(h['added_lines']) for h in f['hunks'])
for f in files
)

for f in files:
for h in f['hunks']:
h.pop('_line_counter', None)

return {
'files': files,
'total_added_lines': total_added,
'_all_hunks': all_hunks,
}
97 changes: 97 additions & 0 deletions examples/skills_code_review_agent/agent/fake_model.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
# Tencent is pleased to support the open source community by making trpc-agent-python available.
# Copyright (C) 2025 Tencent. All rights reserved.
# trpc-agent-python is licensed under the Apache License Version 2.0.
"""Fake model that replays canned responses for Agent-driven code review pipeline.

Enables full Agent testing without real LLM API calls.
"""

from typing import AsyncGenerator, List, Optional

from trpc_agent_sdk.models import LLMModel, LlmRequest, LlmResponse
from trpc_agent_sdk.context import InvocationContext
from trpc_agent_sdk.types import Content, Part


class FakeModel(LLMModel):
"""A fake LLM model that returns pre-recorded agent actions.

Used to drive the LlmAgent through skill_load → skill_run → report
without calling any real LLM API.
"""

def __init__(self, model_name: str = "fake-model", **kwargs):
super().__init__(model_name=model_name, **kwargs)
self._steps: list[LlmResponse] = []
self._step_index = 0

@classmethod
def supported_models(cls) -> List[str]:
return [r"fake-.*", r"mock-.*"]

def set_steps(self, steps: list[LlmResponse]):
"""Configure the sequence of responses this model will return."""
self._steps = steps
self._step_index = 0

async def _generate_async_impl(
self,
request: LlmRequest,
stream: bool = False,
ctx: Optional[InvocationContext] = None,
) -> AsyncGenerator[LlmResponse, None]:
if self._steps and self._step_index < len(self._steps):
resp = self._steps[self._step_index]
self._step_index += 1
yield resp
else:
yield LlmResponse(
content=Content(parts=[Part(text="Review complete.")]),
partial=False,
)

def validate_request(self, request: LlmRequest) -> None:
pass


def build_code_review_steps(diff_path: str) -> list[LlmResponse]:
"""Build a sequence of model responses that drive the code review pipeline."""
return [
LlmResponse(
content=Content(
role="model",
parts=[Part.from_function_call(
name="skill_load",
args={"skill_name": "code-review"}
)]
),
partial=False,
),
LlmResponse(
content=Content(
role="model",
parts=[Part.from_function_call(
name="skill_run",
args={"skill": "code-review", "command": f"python parse_diff.py {diff_path}"}
)]
),
partial=False,
),
LlmResponse(
content=Content(
role="model",
parts=[Part(text=(
"Code review completed. Findings have been stored.\n\n"
"FINDINGS_JSON\n"
"```json\n"
'[{"severity": "high", "category": "security", '
'"file": "a.py", "line": 1, '
'"title": "Hardcoded credential (LLM)", '
'"evidence": "token = \\"abc123\\"", '
'"recommendation": "Use env var", "confidence": 0.9}]\n'
"```"
))]
),
partial=False,
),
]
Loading
Loading