Reviewing 960 applications in a day instead of a month
An application review system built with Claude for an internal CAS program. Around 960 applications scored in about 8 hours, against an estimated 200 hours across 40 reviewers.
Client
CAS
Role
Design and build
Stack
Claude · Structured scoring rubrics · Human-in-the-loop review
- Applications scored
- ~960
- Instead of an estimated 200
- ~8 hrs
- Estimated cost avoided
- ~$23,000
The problem
The program pulls close to a thousand applications in a cycle. The old way pulled about forty people off their real jobs. Two hundred hours between them. Every one of them had to hold the same standard in their head. Application after application.
That's the kind of work worth handing to a model. It repeats. It happens on a schedule. You can write the criteria down. And doing it by hand is slow and uneven.
The approach
The scoring criteria came first. The rubric had to get explicit. Two reviewers should land in about the same place on the same application. That took longer than the build.
Then the system reads each application and scores it against the rubric. It writes out its reasoning for every score instead of handing back a bare number. If you can't see why something got a 3, you can't argue with it. And nobody signs off on a review process they can't check.
People stayed in the loop where it counted. The system does the reading and the first pass. Humans make the calls at the top of the pile and spot-check the rest.
The result
About 960 applications scored in roughly 8 hours. The old process was estimated at 200 hours across 40 reviewers. Call it $23,000 of staff time back, and forty people who didn't lose a week.
Consistency is harder to put a number on. Application 900 gets read the same way as application 4. A tired human reviewer can't really promise that.
Next case study
A reusable Claude Skills library for a marketing team