The most common cause of a poor shortlist isn't bad AI or bad judgment. It's underspecified requirements. When a hiring manager and a recruiter haven't agreed on what good looks like before the search starts, every screening decision becomes a negotiation after the fact.
What makes a requirement structured
A structured requirement has three properties: it's evaluable from evidence a candidate typically provides, it's specific enough that two reviewers would apply it consistently, and it's weighted relative to other criteria so trade-offs can be made explicitly.
An unstructured requirement: "Strong analytical skills." A structured version: "Has demonstrated quantitative analysis in a professional context, such as building a model, running a statistical test, or designing a measurement system, with documented output." The second is harder to write. It's also possible to evaluate against a resume without purely subjective interpretation.
The evaluability test is useful here: before you finalize a requirement, ask whether two independent reviewers reading the same resume would reach the same conclusion about whether the candidate meets it. If the answer is probably not, the requirement isn't structured enough to produce consistent screening. This matters most in the middle of the distribution, where the strong-maybe candidates live. Requirements that are clear enough to rank the obvious top and bottom of the pool are common. Requirements that are specific enough to rank the middle accurately are rare and are where screening quality actually lives.
Why weight matters
Most job descriptions present all requirements as equal. In practice, some requirements are genuinely disqualifying and some are nice-to-have. Without weights, a screening system or a human reviewer can't distinguish between "this candidate is missing one small thing" and "this candidate is missing the core requirement."
In our pilots, the single biggest predictor of hiring manager satisfaction with shortlists was whether requirements were weighted before screening started, not after. When weights are assigned up front, the shortlist reflects what the hiring manager actually cares most about. When weights are assigned retrospectively ("why did you send me this person?"), they reflect post-hoc rationalization.
Weight assignment reveals a second type of underspecification that is less commonly discussed: threshold ambiguity. A requirement like "data analysis experience" might be a disqualifier for a data engineering role but merely a nice-to-have for a business development role. The weight encodes the threshold. Without it, a screening system that finds "data analysis" on a resume has no way to know whether that signal should add 5 points to a score or 30.
The connection between requirements and shortlist size
One underappreciated effect of well-structured requirements is that they produce smaller, higher-quality shortlists naturally. When criteria are specific and weighted, candidates who meet the core requirements clearly score substantially higher than candidates who don't. The top of the ranked list separates from the middle and bottom more cleanly. You end up with 8 candidates who clearly meet the rubric, not 25 who sort of match several requirements to varying degrees.
This matters because shortlist size directly affects hiring manager engagement. A hiring manager who receives 8 well-matched candidates reviews them all carefully. A hiring manager who receives 25 candidates that the recruiter "wasn't sure about" reviews the first 8 carefully and skims the rest. The same screening quality problems that produced the thin shortlist also produced the padded shortlist -- just expressed differently depending on whether the rubric was too strict or too vague.
The alignment conversation
Structured requirements force a conversation that most recruiting processes skip: what does a good hire for this role actually look like? That conversation is uncomfortable because it surfaces disagreements between recruiter and hiring manager, or within the hiring team, about what they're looking for. Having it before a search starts is much better than having it 30 resumes in.
The output of that conversation doesn't need to be elaborate. A rubric with 5 to 8 criteria, each with a one-line description and a weight, is enough. That's the artifact that turns a vague job description into a screening instrument.
A rubric also creates accountability that a JD doesn't. When a hiring manager reviews a shortlist and says "these aren't quite right," the rubric is the shared reference for diagnosing why. Did the candidates meet all the criteria and the criteria were wrong? Did they not meet the criteria clearly enough? Was there a criterion that turned out to matter that wasn't captured? The rubric makes the disagreement specific and resolvable in a way that "I'll know it when I see it" doesn't.
Iterating the rubric across a search
A structured rubric isn't a fixed artifact. It's a living specification that should improve as the search progresses. After the first round of screening, most searches surface one of two signals: either several candidates are bunching near identical scores (which suggests the weighting isn't differentiating enough), or no candidates are clearing a threshold they should plausibly clear (which suggests a criterion is too narrow or the weight is too high relative to the supply in the market). Both are diagnostic, and both are fixable if you can see them in the scoring data.
Rubric iteration is the practice that separates recruiters who get better at a search over time from those who keep sending the same quality shortlist to a frustrated hiring manager. Documenting what changed between rubric versions and why creates a record of the calibration process that has value beyond the current search: it becomes a reference for the next person who opens a similar role, carrying forward what was learned rather than starting from scratch.
