AI · 53m ago
New Benchmark Tests LLMs on Cryptanalysis Tasks
CryptanalysisBench, developed by researchers from ETH Zurich, Anthropic, and others, evaluates LLMs on cryptographic problem-solving across three difficulty tiers. Reported results show 65-86% success on known-break tasks, but under 12 successes on full-strength primitives. The benchmark aims to track AI's evolving capabilities in cryptanalysis without overclaiming current abilities.
Meridian48 take
The benchmark is a useful tool for measuring progress, but the low scores on real-world tasks suggest LLMs are far from posing a practical threat to modern cryptography.
Read the full reporting
CryptanalysisBench Introduces a Framework to Measure LLM Cryptanalysis Capabilities →
DEV Community
llm-benchmarkcryptanalysis