Dev Tools · 1h ago
OpenEval Proposes Standard JSON Format for LLM Evaluation
OpenEval introduces a portable JSON Schema for LLM evaluation test cases, graders, and results, aiming to solve interoperability issues between frameworks like DeepEval and Promptfoo. The project includes TypeScript and Python SDKs, a CLI, and converters for popular frameworks. Version 1.0.0 is now available on npm and PyPI.
Meridian48 take
A practical solution to a real pain point, but adoption will depend on whether major frameworks actually implement the converters.
llm-evaluationopen-source