Is your feature request related to a problem?
The current evaluation process using batchAPI takes up to 24 hours to complete, which slows down our ability to iterate on prompts. Although this method is cost-effective and requires less infrastructure, faster evaluations are needed to enhance workflow.
Describe the solution you'd like
- Implement evaluations for smaller sizes (like 50 or 100) using the response API for quicker results.
- Continue using batchAPI for larger evaluations to maintain cost efficiency.
Is your feature request related to a problem?
The current evaluation process using batchAPI takes up to 24 hours to complete, which slows down our ability to iterate on prompts. Although this method is cost-effective and requires less infrastructure, faster evaluations are needed to enhance workflow.
Describe the solution you'd like