diff --git a/docs/attributes.md b/docs/attributes.md index 620d4469..2d1e9c00 100644 --- a/docs/attributes.md +++ b/docs/attributes.md @@ -3,7 +3,7 @@ layout: page title: Attributes Reference --- -Complete reference for all 25 agent-ready attributes assessed by AgentReady. +Complete reference for all 27 agent-ready attributes assessed by AgentReady.

🤖 Bootstrap Automation

@@ -26,7 +26,7 @@ Complete reference for all 25 agent-ready attributes assessed by AgentReady. ## Overview -AgentReady evaluates repositories against 25 attributes derived from research by Anthropic, Microsoft, Google, ETH Zurich, and Red Hat. Each attribute has specific pass/fail criteria, a tier-based weight, and concrete remediation steps. +AgentReady evaluates repositories against 27 attributes derived from research by Anthropic, Microsoft, Google, ETH Zurich, and Red Hat. Each attribute has specific pass/fail criteria, a tier-based weight, and concrete remediation steps. Each entry below covers: what the assessor checks, the scoring breakdown, and how to fix a failing result. @@ -38,9 +38,9 @@ Attributes are organized into four weighted tiers: | Tier | Weight | Focus | Attribute Count | |------|--------|-------|-----------------| -| **Tier 1: Essential** | 59% | Fundamentals enabling basic AI functionality | 9 attributes | +| **Tier 1: Essential** | 58% | Fundamentals enabling basic AI functionality | 9 attributes | | **Tier 2: Critical** | 27% | Major quality improvements and safety nets | 9 attributes | -| **Tier 3: Important** | 12% | Significant improvements in specific areas | 5 attributes | +| **Tier 3: Important** | 13% | Significant improvements in specific areas | 7 attributes | | **Tier 4: Advanced** | 2% | Refinement and optimization | 2 attributes | Missing a Tier 1 attribute (up to 12% weight) has up to 12x the score impact of missing a Tier 4 attribute (1% weight). @@ -1046,7 +1046,7 @@ setup: ## Tier 3: Important Attributes -*Significant improvements in specific areas — 12% of total score* +*Significant improvements in specific areas — 13% of total score* ### 14. Cyclomatic Complexity Limits @@ -1098,14 +1098,14 @@ radon cc src/ -s -nb ### Additional Tier 3 Attributes -**Structured Logging** (`structured_logging`, 2%) — JSON logs with consistent fields -**OpenAPI/Swagger Specs** (`openapi_specs`, 3%) — Machine-readable API docs +**Structured Logging** (`structured_logging`, 1%) — JSON logs with consistent fields +**OpenAPI/Swagger Specs** (`openapi_specs`, 2%) — Machine-readable API docs **Progressive Disclosure** (`progressive_disclosure`, 2%) — Path-scoped rules, skills for focused context (moved from T4) ### Architecture Decision Records **ID**: `architecture_decisions` **Tier**: Tier 3 -**Weight**: 3% +**Weight**: 2% **Category**: Documentation Standards **Status**: ✅ Implemented @@ -1166,6 +1166,184 @@ EOF --- +### Architectural Boundaries + +**ID**: `architectural_boundaries` +**Tier**: Tier 3 +**Weight**: 2% +**Category**: Repository Structure +**Status**: ✅ Implemented + +#### Definition + +Linter or tooling configuration that enforces module import boundaries, preventing uncontrolled cross-module coupling. As the Factory.ai principle puts it: "agents write code; linters write the law." + +#### Why It Matters + +Without enforced boundaries, AI agents freely import across module lines, creating tight coupling that makes future changes risky. Boundary enforcement via linter rules catches violations at commit time, keeping the architecture intact even when agents generate code at scale. + +#### Measurable Criteria + +**Not applicable** for repos with fewer than 20 files (too small for meaningful module boundaries), or for repos whose detected languages are not yet supported (currently: Python, JavaScript, TypeScript, Go). Repos in Java, Rust, Ruby, C#, and other languages get `not_applicable` rather than being penalized. + +**Binary pass/fail**: the assessor checks for at least one recognized boundary enforcement tool configured in the repository: + +| Tool | Config files checked | Signal | +|------|---------------------|--------| +| ESLint `no-restricted-imports` / `no-restricted-modules` | `.eslintrc.*`, `eslint.config.*`, `package.json` (eslintConfig) | Rule name present in config | +| Go `depguard` / `gomodguard` | `.golangci.yml`, `.golangci.yaml` | Linter name in enabled linters list | +| Python `import-linter` | `.importlinter`, `pyproject.toml` (`[tool.importlinter]`), `setup.cfg` (`[importlinter]`) | Config section present | +| Python `flake8-tidy-imports` | `pyproject.toml`, `setup.cfg` | Plugin or `banned-api` config present | +| `dependency-cruiser` | `.dependency-cruiser.cjs`, `.dependency-cruiser.mjs`, `.dependency-cruiser.js` | Config file exists | + +**Pass**: any one boundary tool configured (score 100). **Fail**: no boundary tools found (score 0). + +#### Remediation + +**JavaScript/TypeScript** (ESLint): + +```json +{ + "rules": { + "no-restricted-imports": ["error", { + "patterns": ["../internal/*"] + }] + } +} +``` + +**Go** (depguard via golangci-lint): + +```yaml +linters: + enable: + - depguard +linters-settings: + depguard: + rules: + main: + deny: + - pkg: "internal/secret" + desc: "Use the public API instead" +``` + +**Python** (import-linter): + +```ini +# .importlinter or [tool.importlinter] in pyproject.toml +[importlinter] +root_packages = myapp +[importlinter:contract:layers] +name = Layered architecture +type = layers +layers = api | service | repository +``` + +**General** (dependency-cruiser): + +```bash +npx depcruise --init +npx depcruise src --config +``` + +**Tools**: ESLint, golangci-lint (depguard/gomodguard), import-linter, flake8-tidy-imports, dependency-cruiser + +**Citations**: + +- Factory.ai: "Agents write code; linters write the law" +- wg-agentic-sdlc: "Hooks and Enforcement" best practices + +--- + +### Threat Model + +**ID**: `threat_model` +**Tier**: Tier 3 +**Weight**: 2% +**Category**: Security +**Status**: ✅ Implemented + +#### Definition + +A structured document (typically `THREAT_MODEL.md`) that identifies system assets, entry points, trust boundaries, and potential threats, following the wg-agentic-sdlc 8-section schema. + +#### Why It Matters + +AI agents generating security-sensitive code need to know what the system is protecting and where the attack surface lies. A threat model gives agents the context to make security-aware decisions: which inputs to validate, which boundaries to respect, and which mitigations to apply. + +#### Measurable Criteria + +**Scoring** (100 point scale): + +| Signal | Points | +|--------|--------| +| Threat model file exists | 40 | +| Substantial content (>500 bytes excluding headings) | 10 | +| Recognized sections (6 pts each, up to 8 sections) | up to 48 | +| Threat table with structured entries | 2 | + +**The 8 canonical sections** (from wg-agentic-sdlc schema): + +1. System context +2. Assets +3. Entry points (and trust boundaries) +4. Threats +5. Deprioritized +6. Open questions +7. Provenance +8. Recommended mitigations + +Section matching accepts numbered (`## 1. System Context`), unnumbered (`## System Context`), and section-symbol-prefixed (`## §7 Adversary model`) headings. Matching uses word-based fuzzy matching against canonical names, plus synonym support for common real-world heading variations (e.g., "Adversary model" matches "threats", "Out of scope" matches "deprioritized", "Trust boundaries" matches "entry points"). + +**Recognized file locations**: `THREAT_MODEL.md`, `THREAT-MODEL.md`, `threat-model.md`, `threat_model.md` at repo root or under `docs/` or `docs/security/`. + +**SECURITY.md fallback**: If no standalone threat model file exists but `SECURITY.md` contains a "threat model" section heading, partial credit (25 pts) is awarded. + +**Pass threshold**: 50 points (file exists with substantial content, or file exists with at least 2 recognized sections). + +#### Remediation + +```bash +cat > THREAT_MODEL.md << 'EOF' +# Threat Model + +## 1. System Context +Describe the system architecture and its environment. + +## 2. Assets +List what the system protects (user data, credentials, etc.). + +## 3. Entry Points +Identify API endpoints, CLI interfaces, file inputs, and trust boundaries. + +## 4. Threats +| Threat | Category | Impact | Likelihood | Mitigation | +|--------|----------|--------|------------|------------| +| SQL injection via search | Injection | High | Medium | Parameterized queries | + +## 5. Deprioritized +Threats considered but deemed low risk for now. + +## 6. Open Questions +Unresolved security questions requiring further investigation. + +## 7. Provenance +How dependencies and build artifacts are verified. + +## 8. Recommended Mitigations +Prioritized list of security improvements to implement. +EOF +``` + +**Tools**: [OWASP Threat Dragon](https://owasp.org/www-project-threat-dragon/), Microsoft Threat Modeling Tool + +**Citations**: + +- wg-agentic-sdlc: "THREAT_MODEL_README.md" schema specification +- OWASP: "Threat Modeling" + +--- + *Full details for each attribute available in the [research document](https://github.com/ambient-code/agentready/blob/main/RESEARCH_REPORT.md).* --- @@ -1185,12 +1363,12 @@ EOF ## Implementation Status -All 24 assessors are fully implemented across all four tiers. +All 27 assessors are fully implemented across all four tiers. **Current State**: - ✅ **Tier 1 (Essential)**: Fully implemented (9 attributes) - ✅ **Tier 2 (Critical)**: Fully implemented (9 attributes) -- ✅ **Tier 3 (Important)**: Fully implemented (5 attributes) +- ✅ **Tier 3 (Important)**: Fully implemented (7 attributes) - ✅ **Tier 4 (Advanced)**: Fully implemented (2 attributes) See the [GitHub repository](https://github.com/ambient-code/agentready) for current implementation details. diff --git a/src/agentready/assessors/__init__.py b/src/agentready/assessors/__init__.py index 904e5763..e5edcaf9 100644 --- a/src/agentready/assessors/__init__.py +++ b/src/agentready/assessors/__init__.py @@ -31,8 +31,9 @@ PatternReferencesAssessor, ProgressiveDisclosureAssessor, ) -from .security import DependencySecurityAssessor +from .security import DependencySecurityAssessor, ThreatModelAssessor from .structure import ( + ArchitecturalBoundaryAssessor, IssuePRTemplatesAssessor, OneCommandSetupAssessor, SeparationOfConcernsAssessor, @@ -60,14 +61,14 @@ def create_all_assessors() -> list[BaseAssessor]: """Create all assessors for assessment. Centralized factory function to eliminate duplication across CLI commands. - Returns all implemented and stub assessors (25 attributes). + Returns all implemented and stub assessors (27 attributes). Returns: List of all assessor instances """ assessors = [ - # Tier 1 Essential — 59% total (9 attributes) - TestExecutionAssessor(), # 12% + # Tier 1 Essential — 58% total (9 attributes) + TestExecutionAssessor(), # 11% TypeAnnotationsAssessor(), # 10% AgentInstructionsAssessor(), # 7% CIQualityGatesAssessor(), # 5% @@ -90,12 +91,14 @@ def create_all_assessors() -> list[BaseAssessor]: DesignIntentAssessor(), # 3% (moved from T3) DbtDataTestsAssessor(), # dbt conditional DbtProjectStructureAssessor(), # dbt conditional - # Tier 3 Important — 12% total (5 attributes) - ArchitectureDecisionsAssessor(), # 3% - OpenAPISpecsAssessor(), # 3% + # Tier 3 Important — 13% total (7 attributes) + ArchitectureDecisionsAssessor(), # 2% + OpenAPISpecsAssessor(), # 2% CyclomaticComplexityAssessor(), # 2% - StructuredLoggingAssessor(), # 2% + StructuredLoggingAssessor(), # 1% ProgressiveDisclosureAssessor(), # 2% (moved from T4) + ArchitecturalBoundaryAssessor(), # 2% (ADR B.1) + ThreatModelAssessor(), # 2% (ADR B.2) # Tier 4 Advanced — 2% total (2 attributes, 1% each) IssuePRTemplatesAssessor(), ContainerSetupAssessor(), diff --git a/src/agentready/assessors/security.py b/src/agentready/assessors/security.py index 0c0564e9..2fb11209 100644 --- a/src/agentready/assessors/security.py +++ b/src/agentready/assessors/security.py @@ -1,6 +1,8 @@ """Security assessors for dependency scanning, SAST, and secret detection.""" import json +import re +from pathlib import Path import yaml @@ -349,3 +351,299 @@ def assess(self, repository: Repository) -> Finding: remediation=remediation, error_message=None, ) + + +class ThreatModelAssessor(BaseAssessor): + """Tier 3 Important - Structured threat model documentation. + + Checks for THREAT_MODEL.md (or equivalent) with structured sections + matching the 8-section schema from the wg-agentic-sdlc best practices. + A threat model enables AI agents to perform focused security scanning + by providing entry points, threat priorities, and scope boundaries. + """ + + CANONICAL_SECTIONS = [ + "system context", + "assets", + "entry points", + "threats", + "deprioritized", + "open questions", + "provenance", + "recommended mitigations", + ] + + SECTION_ALIASES = { + "system context": [ + "scope and", + "intended use", + "system overview", + "architecture overview", + ], + "assets": [ + "sensitive data", + "protected resources", + "data classification", + ], + "entry points": [ + "trust boundaries", + "data flow", + "attack surface", + "interfaces", + ], + "threats": [ + "adversary model", + "threat analysis", + "threat actors", + "risk analysis", + ], + "deprioritized": [ + "out of scope", + "excluded", + "non-threats", + "accepted risks", + ], + "open questions": [ + "unresolved", + "future work", + "open issues", + "known unknowns", + ], + "provenance": [ + "supply chain", + "build integrity", + "software bill", + ], + "recommended mitigations": [ + "controls", + "countermeasures", + "security controls", + "downstream responsibilities", + "remediation", + ], + } + + THREAT_MODEL_FILENAMES = [ + "THREAT_MODEL.md", + "THREAT-MODEL.md", + "threat-model.md", + "threat_model.md", + ] + + THREAT_MODEL_SUBDIRS = [ + "docs", + str(Path("docs") / "security"), + ] + + @property + def attribute_id(self) -> str: + return "threat_model" + + @property + def tier(self) -> int: + return 3 + + @property + def attribute(self) -> Attribute: + return Attribute( + id=self.attribute_id, + name="Threat Model Documentation", + category="Security", + tier=self.tier, + description="Structured THREAT_MODEL.md with security assumptions, attack surface, and prioritized threats", + criteria="THREAT_MODEL.md with recognized section structure (8-section schema)", + default_weight=0.02, + ) + + def assess(self, repository: Repository) -> Finding: + score = 0.0 + evidence = [] + + threat_model_path = self._find_threat_model_file(repository) + + if threat_model_path is None: + fallback_score = self._check_security_md_fallback(repository) + if fallback_score > 0: + return Finding( + attribute=self.attribute, + status="fail", + score=fallback_score, + measured_value="threat model section in SECURITY.md", + threshold="standalone THREAT_MODEL.md with structured sections", + evidence=[ + "No standalone THREAT_MODEL.md found", + "Partial credit: SECURITY.md contains a threat model section", + ], + remediation=self._create_remediation(), + error_message=None, + ) + + return Finding( + attribute=self.attribute, + status="fail", + score=0.0, + measured_value="no threat model found", + threshold="THREAT_MODEL.md with structured sections", + evidence=["No THREAT_MODEL.md or equivalent found"], + remediation=self._create_remediation(), + error_message=None, + ) + + rel_path = threat_model_path.relative_to(repository.path) + score += 40.0 + evidence.append(f"Threat model file found: {rel_path}") + + try: + content = threat_model_path.read_text(encoding="utf-8") + except (OSError, UnicodeDecodeError): + return Finding( + attribute=self.attribute, + status="fail", + score=score, + measured_value=str(rel_path), + threshold="THREAT_MODEL.md with structured sections", + evidence=evidence + ["Could not read file content"], + remediation=self._create_remediation(), + error_message=None, + ) + + non_heading_content = re.sub(r"^#.*$", "", content, flags=re.MULTILINE).strip() + if len(non_heading_content) > 500: + score += 10.0 + evidence.append("Substantial content (>500 bytes)") + + section_count, sections_found = self._count_recognized_sections(content) + if section_count > 0: + section_pts = min(section_count * 6.0, 48.0) + score += section_pts + evidence.append( + f"{section_count}/8 recognized sections: {', '.join(sections_found)}" + ) + + if self._has_threat_table(content): + score += 2.0 + evidence.append("Threats section contains structured table") + + score = min(score, 100.0) + status = "pass" if score >= 50 else "fail" + + return Finding( + attribute=self.attribute, + status=status, + score=score, + measured_value=f"{section_count}/8 sections in {rel_path}", + threshold="THREAT_MODEL.md with structured sections", + evidence=evidence, + remediation=self._create_remediation() if score < 100 else None, + error_message=None, + ) + + def _find_threat_model_file(self, repository: Repository) -> Path | None: + for filename in self.THREAT_MODEL_FILENAMES: + path = repository.path / filename + if path.is_file(): + return path + + for subdir in self.THREAT_MODEL_SUBDIRS: + for filename in self.THREAT_MODEL_FILENAMES: + path = repository.path / subdir / filename + if path.is_file(): + return path + + return None + + def _count_recognized_sections(self, content: str) -> tuple[int, list[str]]: + headings = re.findall(r"^##\s+(?:§?\d+\.?\s*)?(.+)$", content, re.MULTILINE) + found = [] + for heading in headings: + heading_lower = heading.strip().lower() + heading_lower = re.sub(r"\s*[&]\s*", " and ", heading_lower) + matched = self._match_canonical_section(heading_lower) + if matched and matched not in found: + found.append(matched) + return len(found), found + + def _match_canonical_section(self, heading_lower: str) -> str | None: + for canonical in self.CANONICAL_SECTIONS: + canonical_words = canonical.split() + if all(word in heading_lower for word in canonical_words): + return canonical + for canonical, aliases in self.SECTION_ALIASES.items(): + for alias in aliases: + if alias in heading_lower: + return canonical + return None + + def _has_threat_table(self, content: str) -> bool: + threats_match = re.search( + r"^##\s+(?:§?\d+\.?\s*)?(?:threats?|adversary\s+model)\b.*$", + content, + re.MULTILINE | re.IGNORECASE, + ) + if not threats_match: + return False + after_heading = content[threats_match.end() :] + next_section = re.search(r"^##\s+", after_heading, re.MULTILINE) + threats_section = ( + after_heading[: next_section.start()] if next_section else after_heading + ) + return bool(re.search(r"^\|.+\|.+\|", threats_section, re.MULTILINE)) + + def _check_security_md_fallback(self, repository: Repository) -> float: + security_md = repository.path / "SECURITY.md" + if not security_md.exists(): + return 0.0 + try: + content = security_md.read_text(encoding="utf-8") + except (OSError, UnicodeDecodeError): + return 0.0 + if re.search(r"^#+\s+.*threat\s+model", content, re.MULTILINE | re.IGNORECASE): + return 25.0 + return 0.0 + + def _create_remediation(self) -> Remediation: + return Remediation( + summary="Create a THREAT_MODEL.md with structured security analysis", + steps=[ + "Create THREAT_MODEL.md in the repository root", + "Add the 8-section structure: System context, Assets, Entry points, Threats, Deprioritized, Open questions, Provenance, Recommended mitigations", + "Start with system context describing what the project does and its security assumptions", + "List assets (what is worth protecting) with sensitivity levels", + "Document entry points where untrusted input enters the system", + "Add a threat table with actor, impact, likelihood, and status columns", + "Explicitly list deprioritized threats with rationale", + "Point SECURITY.md at the threat model for scope guidance", + ], + tools=[], + commands=[], + examples=[ + "# Threat Model: MyProject\n\n" + "## 1. System context\nA REST API that processes user uploads...\n\n" + "## 2. Assets\n| asset | description | sensitivity |\n|---|---|---|\n" + "| user_data | PII in database | high |\n\n" + "## 3. Entry points & trust boundaries\n| entry_point | description | trust_boundary | reachable_assets |\n" + "|---|---|---|---|\n| /api/upload | File upload endpoint | remote unauth | user_data |\n\n" + "## 4. Threats\n| id | threat | actor | impact | status |\n" + "|---|---|---|---|---|\n| T1 | RCE via file upload | remote_unauth | critical | partially_mitigated |\n\n" + "## 5. Deprioritized\n| threat | reason |\n|---|---|\n" + "| Local file injection | Requires local admin access |\n\n" + "## 6. Open questions\n- Is the upload size limit enforced at the proxy level?\n\n" + "## 7. Provenance\n- mode: bootstrap\n- date: 2026-01-15\n\n" + "## 8. Recommended mitigations\n| mitigation | threat_ids | effort |\n" + "|---|---|---|\n| Sandbox file processing | T1 | M |", + ], + citations=[ + Citation( + source="Red Hat", + title="THREAT_MODEL.md: A checked-in threat model for your repository", + url="", + relevance="Defines the 8-section schema for structured, machine-readable threat models", + ), + Citation( + source="Red Hat", + title="wg-agentic-sdlc Best Practices: Security & Standards", + url="", + relevance="Threat models enable AI agents to perform focused security scanning by providing entry points, threat priorities, and scope boundaries", + ), + ], + ) diff --git a/src/agentready/assessors/structure.py b/src/agentready/assessors/structure.py index 27d35eaa..7e6c4b95 100644 --- a/src/agentready/assessors/structure.py +++ b/src/agentready/assessors/structure.py @@ -1368,3 +1368,245 @@ def _create_remediation(self) -> Remediation: ), ], ) + + +class ArchitecturalBoundaryAssessor(BaseAssessor): + """Assesses whether linter configs enforce module/import boundaries. + + Tier 3 Important (2% weight) - "Agents write code; linters write the law." + Without boundary enforcement, agents freely import across module boundaries + creating coupling that humans would catch in review. + + Returns not_applicable for repos with fewer than 20 files. + """ + + FILE_THRESHOLD = 20 + SUPPORTED_LANGUAGES = {"Python", "JavaScript", "TypeScript", "Go"} + + @property + def attribute_id(self) -> str: + return "architectural_boundaries" + + @property + def tier(self) -> int: + return 3 + + @property + def attribute(self) -> Attribute: + return Attribute( + id=self.attribute_id, + name="Architectural Boundary Lint Rules", + category="Repository Structure", + tier=self.tier, + description="Import restriction rules configured in linter to enforce module boundaries", + criteria="Linter config with import boundary rules (ESLint no-restricted-imports, Go depguard, Python import-linter, or similar)", + default_weight=0.02, + ) + + def _has_supported_language(self, repository: Repository) -> bool: + if not repository.languages: + return True + return bool(set(repository.languages.keys()) & self.SUPPORTED_LANGUAGES) + + def assess(self, repository: Repository) -> Finding: + if repository.total_files < self.FILE_THRESHOLD: + return Finding.not_applicable( + self.attribute, + reason=f"Repository has {repository.total_files} files (boundary rules relevant for >={self.FILE_THRESHOLD})", + ) + + if not self._has_supported_language(repository): + langs = ", ".join(sorted(repository.languages.keys())) + return Finding.not_applicable( + self.attribute, + reason=f"No supported languages detected ({langs}); boundary checks cover Python, JavaScript, TypeScript, Go", + ) + + tools_found = [] + evidence = [] + + self._check_eslint(repository, tools_found, evidence) + self._check_go_boundary_tools(repository, tools_found, evidence) + self._check_python_import_linter(repository, tools_found, evidence) + self._check_dependency_cruiser(repository, tools_found, evidence) + + if tools_found: + return Finding( + attribute=self.attribute, + status="pass", + score=100.0, + measured_value=f"boundary tools: {', '.join(tools_found)}", + threshold="at least one import boundary tool configured", + evidence=evidence, + remediation=None, + error_message=None, + ) + + return Finding( + attribute=self.attribute, + status="fail", + score=0.0, + measured_value="no boundary enforcement found", + threshold="at least one import boundary tool configured", + evidence=evidence or ["No import boundary lint rules detected"], + remediation=self._create_remediation(), + error_message=None, + ) + + def _read_file_safe(self, path: Path) -> str | None: + if not path.exists(): + return None + try: + return path.read_text(encoding="utf-8") + except (OSError, UnicodeDecodeError): + return None + + def _check_eslint( + self, + repository: Repository, + tools_found: list[str], + evidence: list[str], + ) -> None: + eslint_configs = [ + ".eslintrc.json", + ".eslintrc.js", + ".eslintrc.yml", + ".eslintrc.yaml", + "eslint.config.js", + "eslint.config.mjs", + "eslint.config.cjs", + ] + boundary_rules = ["no-restricted-imports", "no-restricted-modules"] + + for config_name in eslint_configs: + content = self._read_file_safe(repository.path / config_name) + if content and any(rule in content for rule in boundary_rules): + matched = [r for r in boundary_rules if r in content] + tools_found.append(f"ESLint ({matched[0]})") + evidence.append(f"ESLint boundary rule in {config_name}: {matched[0]}") + return + + pkg_path = repository.path / "package.json" + content = self._read_file_safe(pkg_path) + if content and "eslintConfig" in content: + if any(rule in content for rule in boundary_rules): + matched = [r for r in boundary_rules if r in content] + tools_found.append(f"ESLint ({matched[0]})") + evidence.append( + f"ESLint boundary rule in package.json eslintConfig: {matched[0]}" + ) + + def _check_go_boundary_tools( + self, + repository: Repository, + tools_found: list[str], + evidence: list[str], + ) -> None: + go_configs = [".golangci.yml", ".golangci.yaml"] + go_tools = ["depguard", "gomodguard"] + + for config_name in go_configs: + content = self._read_file_safe(repository.path / config_name) + if content: + matched = [t for t in go_tools if t in content] + if matched: + tools_found.append(matched[0]) + evidence.append( + f"Go boundary linter in {config_name}: {matched[0]}" + ) + return + + def _check_python_import_linter( + self, + repository: Repository, + tools_found: list[str], + evidence: list[str], + ) -> None: + if (repository.path / ".importlinter").exists(): + tools_found.append("import-linter") + evidence.append("Python import-linter config: .importlinter") + return + + pyproject = self._read_file_safe(repository.path / "pyproject.toml") + if pyproject: + if "[tool.importlinter]" in pyproject: + tools_found.append("import-linter") + evidence.append("Python import-linter config in pyproject.toml") + return + if "flake8-tidy-imports" in pyproject or "banned-api" in pyproject: + tools_found.append("flake8-tidy-imports") + evidence.append( + "Python flake8-tidy-imports/banned-api in pyproject.toml" + ) + return + + setup_cfg = self._read_file_safe(repository.path / "setup.cfg") + if setup_cfg: + if "[importlinter]" in setup_cfg: + tools_found.append("import-linter") + evidence.append("Python import-linter config in setup.cfg") + return + if "flake8-tidy-imports" in setup_cfg or "banned-api" in setup_cfg: + tools_found.append("flake8-tidy-imports") + evidence.append("Python flake8-tidy-imports/banned-api in setup.cfg") + return + + def _check_dependency_cruiser( + self, + repository: Repository, + tools_found: list[str], + evidence: list[str], + ) -> None: + dc_configs = [ + ".dependency-cruiser.cjs", + ".dependency-cruiser.mjs", + ".dependency-cruiser.js", + ] + for config_name in dc_configs: + if (repository.path / config_name).exists(): + tools_found.append("dependency-cruiser") + evidence.append(f"dependency-cruiser config: {config_name}") + return + + def _create_remediation(self) -> Remediation: + return Remediation( + summary="Configure import boundary rules in your linter", + steps=[ + "Identify module boundaries in your codebase (e.g., frontend vs backend, domain vs infrastructure)", + "Add import restriction rules to your existing linter configuration", + "For JavaScript/TypeScript: add ESLint no-restricted-imports rule", + "For Go: enable depguard or gomodguard in .golangci.yml", + "For Python: configure import-linter or flake8-tidy-imports", + "For any language: consider dependency-cruiser for cross-language boundary enforcement", + ], + tools=[ + "ESLint", + "golangci-lint", + "import-linter", + "dependency-cruiser", + ], + commands=[ + "# Python: pip install import-linter", + "# Go: add depguard to .golangci.yml linters list", + "# JS/TS: add no-restricted-imports to ESLint rules", + "# Any: npx dependency-cruiser --init", + ], + examples=[ + '# ESLint (.eslintrc.json)\n"rules": {\n "no-restricted-imports": ["error", {\n "patterns": [{\n "group": ["../backend/*"],\n "message": "Frontend cannot import backend modules directly"\n }]\n }]\n}', + "# Go (.golangci.yml)\nlinters:\n enable:\n - depguard\nlinters-settings:\n depguard:\n rules:\n main:\n deny:\n - pkg: internal/\n desc: Use public API instead of internal packages", + ], + citations=[ + Citation( + source="Factory.ai", + title="Using Linters to Direct Agents", + url="https://factory.ai/news/using-linters-to-direct-agents", + relevance="'Agents write code; linters write the law' principle for boundary enforcement", + ), + Citation( + source="Red Hat", + title="Repository Scaffolding for AI Coding Agents, Section 3.3", + url="", + relevance="Architectural boundary lint rules as Tier 3 best practice", + ), + ], + ) diff --git a/src/agentready/data/default-weights.yaml b/src/agentready/data/default-weights.yaml index ee985a03..ebb87a4d 100644 --- a/src/agentready/data/default-weights.yaml +++ b/src/agentready/data/default-weights.yaml @@ -1,15 +1,15 @@ # Default Tier-Based Weight Distribution # -# This file defines the default weights for all 25 attributes. +# This file defines the default weights for all 27 attributes. # Weights are based on evidence from ETH Zurich (Feb 2026), Anthropic, # Red Hat best practices (April 2026), and Cursor agent guidelines. # # Key finding: Verification (tests, lint, CI) has highest impact on # agent effectiveness. Documentation helps only marginally (+4%). # -# Tier 1 (Essential): 59% total (9 attributes) +# Tier 1 (Essential): 58% total (9 attributes) # Tier 2 (Critical): 27% total (3% each, 9 attributes) -# Tier 3 (Important): 12% total (varies, 5 attributes) +# Tier 3 (Important): 13% total (varies, 7 attributes) # Tier 4 (Advanced): 2% total (1% each, 2 attributes) # TOTAL: 100% (sum to 1.0) # @@ -24,9 +24,15 @@ # where agent readiness matters most # - Redistributed freed 4% to test_execution (+2%) and # type_annotations (+2%), the highest-evidence T1 attributes +# +# Changes in v2.2.0 (ADR Proposal B: new assessors): +# - Added architectural_boundaries (T3, 2%): import boundary lint rules +# - Added threat_model (T3, 2%): structured threat model documentation +# - Reduced test_execution (12% -> 11%), architecture_decisions (3% -> 2%), +# structured_logging (2% -> 1%), openapi_specs (3% -> 2%) to fund new weights -# Tier 1 (Essential) - 59% total weight -test_execution: 0.12 # 5.1 - Test Execution & Coverage +# Tier 1 (Essential) - 58% total weight +test_execution: 0.11 # 5.1 - Test Execution & Coverage type_annotations: 0.10 # 3.3 - Type Annotations (structural signals for agents) agent_instructions: 0.07 # 1.1 - Agent Instruction Files (CLAUDE.md/AGENTS.md) ci_quality_gates: 0.05 # 16.2 - CI Quality Gates (lint + type-check + tests on PR) @@ -47,12 +53,14 @@ separation_of_concerns: 0.03 # 4.2 - Separation of Concerns pattern_references: 0.03 # 17.1 - Pattern References for Common Changes design_intent: 0.03 # 17.2 - Design Intent Documentation (moved from T3) -# Tier 3 (Important) - 12% total weight (5 attributes) -architecture_decisions: 0.03 # 2.3 - Architecture Decision Records -openapi_specs: 0.03 # 10.1 - OpenAPI/Swagger Specifications +# Tier 3 (Important) - 13% total weight (7 attributes) +architecture_decisions: 0.02 # 2.3 - Architecture Decision Records +openapi_specs: 0.02 # 10.1 - OpenAPI/Swagger Specifications cyclomatic_complexity: 0.02 # 3.1 - Cyclomatic Complexity Thresholds -structured_logging: 0.02 # 9.2 - Structured Logging +structured_logging: 0.01 # 9.2 - Structured Logging progressive_disclosure: 0.02 # 17.3 - Progressive Disclosure (moved from T4) +architectural_boundaries: 0.02 # B.1 - Architectural Boundary Lint Rules (ADR) +threat_model: 0.02 # B.2 - Threat Model Documentation (ADR) # Tier 4 (Advanced) - 2% total weight issue_pr_templates: 0.01 # 7.3 - Issue & Pull Request Templates diff --git a/tests/unit/test_assessors_security.py b/tests/unit/test_assessors_security.py index 9c1c3884..8fedfcba 100644 --- a/tests/unit/test_assessors_security.py +++ b/tests/unit/test_assessors_security.py @@ -2,7 +2,10 @@ import subprocess -from agentready.assessors.security import DependencySecurityAssessor +from agentready.assessors.security import ( + DependencySecurityAssessor, + ThreatModelAssessor, +) from agentready.models.repository import Repository @@ -838,3 +841,348 @@ def test_renovate_json5_with_meaningful_package_json_fallback(self, tmp_path): assert any( "Meaningful Renovate configuration detected" in e for e in finding.evidence ) + + +class TestThreatModelAssessor: + """Test ThreatModelAssessor (ADR B.2).""" + + def _make_repo(self, tmp_path, **kwargs): + (tmp_path / ".git").mkdir(exist_ok=True) + defaults = dict( + path=tmp_path, + name="test-repo", + url=None, + branch="main", + commit_hash="abc123", + languages={"Python": 100}, + total_files=50, + total_lines=5000, + ) + defaults.update(kwargs) + return Repository(**defaults) + + FULL_THREAT_MODEL = """# Threat Model: TestApp + +## 1. System context +A REST API that processes user data. + +## 2. Assets +| asset | description | sensitivity | +|---|---|---| +| user_data | PII in database | high | + +## 3. Entry points & trust boundaries +| entry_point | description | trust_boundary | reachable_assets | +|---|---|---|---| +| /api/upload | File upload | remote unauth | user_data | + +## 4. Threats +| id | threat | actor | impact | status | +|---|---|---|---|---| +| T1 | RCE via upload | remote_unauth | critical | partially_mitigated | + +## 5. Deprioritized +| threat | reason | +|---|---| +| Local file injection | Requires admin | + +## 6. Open questions +- Is size limit enforced at proxy? + +## 7. Provenance +- mode: bootstrap +- date: 2026-01-15 + +## 8. Recommended mitigations +| mitigation | threat_ids | effort | +|---|---|---| +| Sandbox processing | T1 | M | +""" + + def test_no_threat_model_fails(self, tmp_path): + """No threat model file returns fail with score 0.""" + repo = self._make_repo(tmp_path) + assessor = ThreatModelAssessor() + finding = assessor.assess(repo) + assert finding.status == "fail" + assert finding.score == 0.0 + assert finding.remediation is not None + + def test_empty_threat_model(self, tmp_path): + """Empty THREAT_MODEL.md gets existence credit only.""" + (tmp_path / "THREAT_MODEL.md").write_text("") + repo = self._make_repo(tmp_path) + assessor = ThreatModelAssessor() + finding = assessor.assess(repo) + assert finding.status == "fail" + assert finding.score == 40.0 + + def test_full_threat_model_all_sections(self, tmp_path): + """THREAT_MODEL.md with all 8 sections scores 100.""" + (tmp_path / "THREAT_MODEL.md").write_text(self.FULL_THREAT_MODEL) + repo = self._make_repo(tmp_path) + assessor = ThreatModelAssessor() + finding = assessor.assess(repo) + assert finding.status == "pass" + assert finding.score == 100.0 + assert "8/8" in finding.measured_value + + def test_partial_sections(self, tmp_path): + """THREAT_MODEL.md with 4/8 sections gets proportional credit.""" + content = """# Threat Model + +## 1. System context +A web application. + +## 2. Assets +User data. + +## 4. Threats +| id | threat | actor | impact | status | +|---|---|---|---|---| +| T1 | XSS | remote_unauth | high | unmitigated | + +## 6. Open questions +- Need to review auth flow. +""" + (tmp_path / "THREAT_MODEL.md").write_text(content) + repo = self._make_repo(tmp_path) + assessor = ThreatModelAssessor() + finding = assessor.assess(repo) + assert finding.status == "pass" + assert 60 <= finding.score <= 80 + assert "4/8" in finding.measured_value + + def test_substantial_content_bonus(self, tmp_path): + """File with >500 bytes of non-heading content gets bonus.""" + content = "# Threat Model\n\n" + "This is a detailed threat analysis. " * 30 + (tmp_path / "THREAT_MODEL.md").write_text(content) + repo = self._make_repo(tmp_path) + assessor = ThreatModelAssessor() + finding = assessor.assess(repo) + assert finding.score == 50.0 # 40 existence + 10 content + + def test_alternative_filename_threat_dash_model(self, tmp_path): + """threat-model.md (lowercase, hyphenated) is detected.""" + (tmp_path / "threat-model.md").write_text( + "# Threat Model\n\n## 1. System context\nA CLI tool.\n" + ) + repo = self._make_repo(tmp_path) + assessor = ThreatModelAssessor() + finding = assessor.assess(repo) + assert finding.status == "fail" # Only 1 section + existence = 46 + assert finding.score >= 40 + + def test_docs_subdirectory(self, tmp_path): + """THREAT_MODEL.md in docs/ is detected.""" + docs = tmp_path / "docs" + docs.mkdir() + (docs / "THREAT_MODEL.md").write_text(self.FULL_THREAT_MODEL) + repo = self._make_repo(tmp_path) + assessor = ThreatModelAssessor() + finding = assessor.assess(repo) + assert finding.status == "pass" + assert "docs/THREAT_MODEL.md" in finding.measured_value + + def test_docs_security_subdirectory(self, tmp_path): + """THREAT_MODEL.md in docs/security/ is detected.""" + sec_dir = tmp_path / "docs" / "security" + sec_dir.mkdir(parents=True) + (sec_dir / "THREAT_MODEL.md").write_text(self.FULL_THREAT_MODEL) + repo = self._make_repo(tmp_path) + assessor = ThreatModelAssessor() + finding = assessor.assess(repo) + assert finding.status == "pass" + + def test_security_md_fallback(self, tmp_path): + """SECURITY.md with threat model heading gives partial credit.""" + (tmp_path / "SECURITY.md").write_text( + "# Security Policy\n\n## Threat Model\nWe consider the following threats...\n" + ) + repo = self._make_repo(tmp_path) + assessor = ThreatModelAssessor() + finding = assessor.assess(repo) + assert finding.status == "fail" + assert finding.score == 25.0 + assert any("SECURITY.md" in e for e in finding.evidence) + + def test_security_md_no_threat_section(self, tmp_path): + """SECURITY.md without threat model heading gives no credit.""" + (tmp_path / "SECURITY.md").write_text( + "# Security Policy\n\nReport vulnerabilities to security@example.com\n" + ) + repo = self._make_repo(tmp_path) + assessor = ThreatModelAssessor() + finding = assessor.assess(repo) + assert finding.score == 0.0 + + def test_unnumbered_sections(self, tmp_path): + """Sections without numbers are recognized.""" + content = """# Threat Model + +## System context +A microservice. + +## Assets +| asset | sensitivity | +|---|---| +| tokens | critical | + +## Threats +| id | threat | impact | +|---|---|---| +| T1 | Token theft | critical | +""" + (tmp_path / "THREAT_MODEL.md").write_text(content) + repo = self._make_repo(tmp_path) + assessor = ThreatModelAssessor() + finding = assessor.assess(repo) + assert finding.status == "pass" + assert "3/8" in finding.measured_value + + def test_threat_table_detection(self, tmp_path): + """Table in threats section gives bonus points.""" + content = """# Threat Model + +## 4. Threats +| id | threat | actor | impact | +|---|---|---|---| +| T1 | Injection | remote_unauth | high | +""" + (tmp_path / "THREAT_MODEL.md").write_text(content) + repo = self._make_repo(tmp_path) + assessor = ThreatModelAssessor() + finding = assessor.assess(repo) + assert any("table" in e.lower() for e in finding.evidence) + + def test_malformed_file_no_crash(self, tmp_path): + """Malformed file doesn't crash, returns fail (below pass threshold).""" + (tmp_path / "THREAT_MODEL.md").write_bytes(b"\xff\xfe bad encoding") + repo = self._make_repo(tmp_path) + assessor = ThreatModelAssessor() + finding = assessor.assess(repo) + assert finding.status == "fail" + assert finding.score == 40.0 + + def test_attribute_properties(self): + """Verify attribute metadata.""" + assessor = ThreatModelAssessor() + assert assessor.attribute_id == "threat_model" + assert assessor.tier == 3 + assert assessor.attribute.default_weight == 0.02 + + def test_root_file_takes_precedence_over_docs(self, tmp_path): + """Root THREAT_MODEL.md is found before docs/ variant.""" + (tmp_path / "THREAT_MODEL.md").write_text( + "# Threat Model\n\n## 1. System context\nRoot file.\n" + ) + docs = tmp_path / "docs" + docs.mkdir() + (docs / "THREAT_MODEL.md").write_text(self.FULL_THREAT_MODEL) + repo = self._make_repo(tmp_path) + assessor = ThreatModelAssessor() + finding = assessor.assess(repo) + assert "THREAT_MODEL.md" in finding.measured_value + assert "docs" not in finding.measured_value + + def test_section_symbol_prefix(self, tmp_path): + """Headings with section symbol prefix are recognized.""" + content = ( + "# Threat Model\n\n" + "## §7 Adversary model\n" + "| threat | impact |\n|---|---|\n| RCE | high |\n\n" + "## §14 Open questions\n" + "What about X?\n" + ) + (tmp_path / "THREAT_MODEL.md").write_text(content) + repo = self._make_repo(tmp_path) + assessor = ThreatModelAssessor() + finding = assessor.assess(repo) + assert any( + "threats" in s + for s in finding.evidence[2] + if isinstance(finding.evidence[2], str) + ) or "2/8" in str(finding.evidence) + sections_evidence = [ + e for e in finding.evidence if "recognized sections" in e.lower() + ] + assert len(sections_evidence) == 1 + assert "threats" in sections_evidence[0] + assert "open questions" in sections_evidence[0] + + def test_synonym_matching(self, tmp_path): + """Alias headings match their canonical equivalents.""" + content = ( + "# Threat Model\n\n" + "## Scope and intended use\n" + "A REST API.\n\n" + "## Trust boundaries and data flow\n" + "External users access via /api.\n\n" + "## Out of scope\n" + "Local attacks.\n\n" + "## Adversary model\n" + "Remote unauthenticated attackers.\n" + ) + (tmp_path / "THREAT_MODEL.md").write_text(content) + repo = self._make_repo(tmp_path) + assessor = ThreatModelAssessor() + finding = assessor.assess(repo) + sections_evidence = [ + e for e in finding.evidence if "recognized sections" in e.lower() + ] + assert len(sections_evidence) == 1 + matched = sections_evidence[0] + assert "system context" in matched + assert "entry points" in matched + assert "deprioritized" in matched + assert "threats" in matched + + def test_ranger_style_threat_model(self, tmp_path): + """Ranger-style threat model with section symbols scores well.""" + content = ( + "# Threat Model: Apache Ranger\n\n" + "## §1 Header\n" + "Provenance and maintainer info.\n" + "Reviewed by PMC.\n\n" + "## §2 Scope and intended use\n" + "Authorization framework for Hadoop ecosystem.\n" + "Detailed system context here with substantial content " + "that exceeds the 500 byte threshold for content bonus. " * 10 + "\n\n" + "## §3 Out of scope\n" + "Local kernel exploits.\n\n" + "## §4 Trust boundaries and data flow\n" + "Admin UI, REST API, plugin interface.\n\n" + "## §5 Protected resources\n" + "Policy data, audit logs, encryption keys.\n\n" + "## §6 Assumptions about inputs\n" + "All inputs validated at API boundary.\n\n" + "## §7 Adversary model\n" + "| threat | actor | impact | status |\n" + "|---|---|---|---|\n" + "| Privilege escalation | authenticated user | high | mitigated |\n\n" + "## §10 Downstream responsibilities\n" + "Plugin authors must validate inputs.\n\n" + "## §14 Open questions\n" + "Key rotation frequency.\n" + ) + (tmp_path / "THREAT_MODEL.md").write_text(content) + repo = self._make_repo(tmp_path) + assessor = ThreatModelAssessor() + finding = assessor.assess(repo) + assert finding.status == "pass" + assert finding.score >= 80 + + def test_adversary_model_table_detection(self, tmp_path): + """Threat table bonus triggers for adversary model heading.""" + content = ( + "# Threat Model\n\n" + "## Adversary model\n" + "| threat | actor | impact |\n" + "|---|---|---|\n" + "| SQLi | remote | high |\n" + ) + (tmp_path / "THREAT_MODEL.md").write_text(content) + repo = self._make_repo(tmp_path) + assessor = ThreatModelAssessor() + finding = assessor.assess(repo) + assert any("table" in e.lower() for e in finding.evidence) diff --git a/tests/unit/test_assessors_structure.py b/tests/unit/test_assessors_structure.py index 015ec50a..5c8d2d11 100644 --- a/tests/unit/test_assessors_structure.py +++ b/tests/unit/test_assessors_structure.py @@ -5,6 +5,7 @@ import pytest from agentready.assessors.structure import ( + ArchitecturalBoundaryAssessor, IssuePRTemplatesAssessor, StandardLayoutAssessor, ) @@ -1334,3 +1335,200 @@ def test_org_fallback_non_github_url(self, assessor, tmp_path): assert finding.status == "fail" assert finding.score == 0 + + +class TestArchitecturalBoundaryAssessor: + """Test ArchitecturalBoundaryAssessor (ADR B.1).""" + + def _make_repo(self, tmp_path, total_files=50, **kwargs): + (tmp_path / ".git").mkdir(exist_ok=True) + defaults = dict( + path=tmp_path, + name="test-repo", + url=None, + branch="main", + commit_hash="abc123", + languages={"Python": 100}, + total_files=total_files, + total_lines=5000, + ) + defaults.update(kwargs) + return Repository(**defaults) + + def test_small_repo_not_applicable(self, tmp_path): + """Repos with <20 files return not_applicable.""" + repo = self._make_repo(tmp_path, total_files=15) + assessor = ArchitecturalBoundaryAssessor() + finding = assessor.assess(repo) + assert finding.status == "not_applicable" + assert "15 files" in finding.evidence[0] + + def test_no_boundary_tools_fails(self, tmp_path): + """Repo with no boundary enforcement tools fails.""" + repo = self._make_repo(tmp_path) + assessor = ArchitecturalBoundaryAssessor() + finding = assessor.assess(repo) + assert finding.status == "fail" + assert finding.score == 0.0 + assert finding.remediation is not None + + def test_eslint_no_restricted_imports(self, tmp_path): + """ESLint no-restricted-imports rule detected.""" + (tmp_path / ".eslintrc.json").write_text( + '{"rules": {"no-restricted-imports": ["error", {"patterns": ["../backend/*"]}]}}' + ) + repo = self._make_repo(tmp_path, languages={"TypeScript": 100}) + assessor = ArchitecturalBoundaryAssessor() + finding = assessor.assess(repo) + assert finding.status == "pass" + assert finding.score == 100.0 + assert any("no-restricted-imports" in e for e in finding.evidence) + + def test_eslint_no_restricted_modules(self, tmp_path): + """ESLint no-restricted-modules rule detected.""" + (tmp_path / ".eslintrc.yml").write_text( + "rules:\n no-restricted-modules:\n - error\n - fs" + ) + repo = self._make_repo(tmp_path, languages={"JavaScript": 100}) + assessor = ArchitecturalBoundaryAssessor() + finding = assessor.assess(repo) + assert finding.status == "pass" + assert finding.score == 100.0 + + def test_eslint_flat_config(self, tmp_path): + """ESLint flat config (eslint.config.js) detected.""" + (tmp_path / "eslint.config.js").write_text( + 'export default [{ rules: { "no-restricted-imports": "error" } }]' + ) + repo = self._make_repo(tmp_path, languages={"TypeScript": 100}) + assessor = ArchitecturalBoundaryAssessor() + finding = assessor.assess(repo) + assert finding.status == "pass" + assert finding.score == 100.0 + + def test_eslint_in_package_json(self, tmp_path): + """ESLint config in package.json eslintConfig detected.""" + (tmp_path / "package.json").write_text( + '{"eslintConfig": {"rules": {"no-restricted-imports": "error"}}}' + ) + repo = self._make_repo(tmp_path, languages={"JavaScript": 100}) + assessor = ArchitecturalBoundaryAssessor() + finding = assessor.assess(repo) + assert finding.status == "pass" + assert "package.json" in finding.evidence[0] + + def test_go_depguard(self, tmp_path): + """Go depguard in golangci config detected.""" + (tmp_path / ".golangci.yml").write_text("linters:\n enable:\n - depguard\n") + repo = self._make_repo(tmp_path, languages={"Go": 100}) + assessor = ArchitecturalBoundaryAssessor() + finding = assessor.assess(repo) + assert finding.status == "pass" + assert finding.score == 100.0 + assert any("depguard" in e for e in finding.evidence) + + def test_go_gomodguard(self, tmp_path): + """Go gomodguard in golangci config detected.""" + (tmp_path / ".golangci.yaml").write_text( + "linters:\n enable:\n - gomodguard\n" + ) + repo = self._make_repo(tmp_path, languages={"Go": 100}) + assessor = ArchitecturalBoundaryAssessor() + finding = assessor.assess(repo) + assert finding.status == "pass" + assert any("gomodguard" in e for e in finding.evidence) + + def test_python_importlinter_file(self, tmp_path): + """Python .importlinter config file detected.""" + (tmp_path / ".importlinter").write_text("[importlinter]\nroot_package=myapp\n") + repo = self._make_repo(tmp_path) + assessor = ArchitecturalBoundaryAssessor() + finding = assessor.assess(repo) + assert finding.status == "pass" + assert any("import-linter" in e for e in finding.evidence) + + def test_python_importlinter_in_pyproject(self, tmp_path): + """Python import-linter in pyproject.toml detected.""" + (tmp_path / "pyproject.toml").write_text( + "[tool.importlinter]\nroot_packages = ['myapp']\n" + ) + repo = self._make_repo(tmp_path) + assessor = ArchitecturalBoundaryAssessor() + finding = assessor.assess(repo) + assert finding.status == "pass" + assert any("pyproject.toml" in e for e in finding.evidence) + + def test_python_importlinter_in_setup_cfg(self, tmp_path): + """Python import-linter in setup.cfg detected.""" + (tmp_path / "setup.cfg").write_text("[importlinter]\nroot_package = myapp\n") + repo = self._make_repo(tmp_path) + assessor = ArchitecturalBoundaryAssessor() + finding = assessor.assess(repo) + assert finding.status == "pass" + assert any("setup.cfg" in e for e in finding.evidence) + + def test_python_flake8_tidy_imports(self, tmp_path): + """Python flake8-tidy-imports in pyproject.toml detected.""" + (tmp_path / "pyproject.toml").write_text( + '[tool.ruff]\nextend-select = ["TID"]\n[tool.ruff.flake8-tidy-imports]\n' + ) + repo = self._make_repo(tmp_path) + assessor = ArchitecturalBoundaryAssessor() + finding = assessor.assess(repo) + assert finding.status == "pass" + assert any("flake8-tidy-imports" in e for e in finding.evidence) + + def test_dependency_cruiser(self, tmp_path): + """dependency-cruiser config detected.""" + (tmp_path / ".dependency-cruiser.cjs").write_text("module.exports = {}") + repo = self._make_repo(tmp_path, languages={"JavaScript": 100}) + assessor = ArchitecturalBoundaryAssessor() + finding = assessor.assess(repo) + assert finding.status == "pass" + assert any("dependency-cruiser" in e for e in finding.evidence) + + def test_malformed_config_no_crash(self, tmp_path): + """Malformed config files don't crash the assessor.""" + (tmp_path / ".eslintrc.json").write_bytes(b"\xff\xfe bad encoding") + repo = self._make_repo(tmp_path) + assessor = ArchitecturalBoundaryAssessor() + finding = assessor.assess(repo) + assert finding.status == "fail" + assert finding.score == 0.0 + + def test_eslint_config_without_boundary_rules(self, tmp_path): + """ESLint config without boundary rules does not pass.""" + (tmp_path / ".eslintrc.json").write_text('{"rules": {"no-console": "error"}}') + repo = self._make_repo(tmp_path, languages={"JavaScript": 100}) + assessor = ArchitecturalBoundaryAssessor() + finding = assessor.assess(repo) + assert finding.status == "fail" + assert finding.score == 0.0 + + def test_attribute_properties(self): + """Verify attribute metadata.""" + assessor = ArchitecturalBoundaryAssessor() + assert assessor.attribute_id == "architectural_boundaries" + assert assessor.tier == 3 + assert assessor.attribute.default_weight == 0.02 + + def test_java_repo_not_applicable(self, tmp_path): + """Java-only repo gets not_applicable (unsupported language).""" + repo = self._make_repo(tmp_path, languages={"Java": 90, "XML": 10}) + assessor = ArchitecturalBoundaryAssessor() + finding = assessor.assess(repo) + assert finding.status == "not_applicable" + + def test_mixed_language_with_supported(self, tmp_path): + """Repo with Java and Python is still applicable.""" + repo = self._make_repo(tmp_path, languages={"Java": 60, "Python": 40}) + assessor = ArchitecturalBoundaryAssessor() + finding = assessor.assess(repo) + assert finding.status != "not_applicable" + + def test_no_languages_defaults_applicable(self, tmp_path): + """Repo with empty languages dict remains applicable.""" + repo = self._make_repo(tmp_path, languages={}) + assessor = ArchitecturalBoundaryAssessor() + finding = assessor.assess(repo) + assert finding.status != "not_applicable"