Surge AI

Get the latest jobs and industry news

Real software engineering happens in repositories with years of accumulated history. Legacy code nobody wants to touch. Historical accidents that became architectural decisions. Dependencies that break in surprising ways. That’s why most sudo L7 tasks are set in privately licensed production codebases from real companies. The tasks are written by staff and principal engineers, former CTOs, and people who’ve built production infrastructure at places like Google, Meta, GitHub, and NASA JPL. Every task starts from a meaningful failure observed in a frontier model. Each one is graded on eight dimensions, from functional correctness to knowing when to push back on a flawed request. sudo L7 measures the judgment that separates a capable coder from a staff-level engineer.

Introducing sudo L7: Can Coding Agents Act Like Staff Engineers? Coding agents are getting very good at solving well-defined problems. But staff engineers do much more than write correct code. They understand unfamiliar systems, make architectural decisions, spot risks the ticket never mentioned, and know when passing the tests isn't enough. sudo L7 measures that judgment. Our new software engineering benchmark contains 60 expert-authored tasks, primarily in private production codebases from real companies, complete with legacy code, technical debt, and messy dependencies. We evaluate eight dimensions of engineering behavior, from functional correctness and engineering craft to practical judgment, thought partnership, and knowing when to push back. The best-performing agents score under ~45%. Coding is only part of the job. See you next cycle. Explore sudo L7: https://lnkd.in/grmJpA4J

Frontier models are available to everyone. Your workflows, expertise, standards, and institutional knowledge are not. Surge AI builds training, benchmark, and evaluation data with vetted experts in your domain, backed by the expertise that made us a trusted data partner to the world's leading AI labs. We find what's limiting your AI's performance, then build the data to fix it, with your team. Start a pilot and see results in 1 week.

Frontier models can already do remarkable work. But an ordinary workday is a different test. Tuesday measures the broad range of professional intelligence, built from the Surge benchmark suite.