Issues

• Run the 5-minute quickstart and report where you got stuck

  good first issue
  buildinghumanetech/humanebench#116 · Filter by author ErikaOnFire opened on Oct 8, 2026

• Translate five scenarios into your language

  good first issue
  buildinghumanetech/humanebench#115 · Filter by author ErikaOnFire opened on Oct 8, 2026

• Rate a batch of model responses (no code needed)

  good first issue
  buildinghumanetech/humanebench#114 · Filter by author ErikaOnFire opened on Oct 8, 2026

• Epic: make the HumaneBench CLI the primary harness and its numbers defensible

  accountable:Jackenhancement
  buildinghumanetech/humanebench#102 · Filter by author jacksenechal opened on Sep 11, 2026
  2

• formatPrompt: two-pass replace lets payload content capture the second placeholder

  buildinghumanetech/humanebench#96 · Filter by author ErikaOnFire opened on Sep 9, 2026

• formatPrompt: $-patterns in evaluation payloads silently corrupt the judge prompt

  buildinghumanetech/humanebench#95 · Filter by author ErikaOnFire opened on Sep 9, 2026

• Support re-judging external prompt/response datasets

  buildinghumanetech/humanebench#89 · Filter by author andalibmalit opened on Aug 20, 2026

• Enable and enforce variable vulnerable-populations percentage for generated rows

  buildinghumanetech/humanebench#36 · Filter by author andalibmalit opened on Oct 21, 2025
  

• Manual dataset population

  accountable:Andalibenhancementgood first issue
  buildinghumanetech/humanebench#25 · Filter by author andalibmalit opened on Oct 3, 2025 · Initial research
  1

  

• Create automated test suite for benchmarking logic

  accountable:Jackenhancementgood first issue
  buildinghumanetech/humanebench#15 · Filter by author andalibmalit opened on Sep 29, 2025 · Improve evaluation logic for HumaneBench
  1

• Create automated test suite for data generation pipeline

  accountable:Andalibenhancementgood first issue
  buildinghumanetech/humanebench#14 · Filter by author andalibmalit opened on Sep 29, 2025 · Improve HumaneBench Synthetic Dataset Pipeline
  2

• Configurable Pipeline Prompting

  accountable:Andalibenhancement
  buildinghumanetech/humanebench#13 · Filter by author andalibmalit opened on Sep 29, 2025 · Improve HumaneBench Synthetic Dataset Pipeline

• MVP of Quality Assurance Tools

  accountable:Andalibenhancement
  buildinghumanetech/humanebench#12 · Filter by author andalibmalit opened on Sep 29, 2025 · Improve HumaneBench Synthetic Dataset Pipeline

• Enhanced Semantic Deduplication at Scale

  accountable:Andalibenhancement
  buildinghumanetech/humanebench#11 · Filter by author andalibmalit opened on Sep 28, 2025 · Improve HumaneBench Synthetic Dataset Pipeline

• Add Distribution Control Engine

  accountable:Andalibenhancement
  buildinghumanetech/humanebench#10 · Filter by author andalibmalit opened on Sep 28, 2025 · Improve HumaneBench Synthetic Dataset Pipeline

• Implement Pydantic Schema Validation

  accountable:Andalibenhancementgood first issue
  buildinghumanetech/humanebench#6 · Filter by author andalibmalit opened on Sep 28, 2025 · Improve HumaneBench Synthetic Dataset Pipeline

• Create SDK for application/vertical-specific humane benchmarking

  accountable:Jackenhancement
  buildinghumanetech/humanebench#5 · Filter by author andalibmalit opened on Sep 28, 2025 · Improve evaluation logic for HumaneBench
  2

• Rich Eval Metadata Collection

  accountable:Jackenhancement
  buildinghumanetech/humanebench#4 · Filter by author andalibmalit opened on Sep 28, 2025 · Improve evaluation logic for HumaneBench

• Custom Metrics and Aggregation

  accountable:Jackenhancement
  buildinghumanetech/humanebench#3 · Filter by author andalibmalit opened on Sep 28, 2025 · Improve evaluation logic for HumaneBench

• Resilient Scoring System

  accountable:Jackenhancement
  buildinghumanetech/humanebench#1 · Filter by author andalibmalit opened on Sep 28, 2025 · Improve evaluation logic for HumaneBench