Evidence to Validate an Executive Search Firm Shortlist for Specialized Commercial Roles

What must shortlist evidence prove?

Evidence must prove the firm can do this specific search. Four things carry the decision: capability in the exact role and reach into the candidate pool you care about, plus execution discipline you can inspect and control of the risks that sink searches quietly. Brand and league tables answer none of that.

The stakes justify the scrutiny. An analysis by Heidrick & Struggles covering 20,000 executive placements found roughly 40% failed within 18 months, with cultural misalignment and unclear role scope among the recurring causes.

Read that number carefully, because it reframes what you're buying. If two out of five senior hires come apart inside a year and a half, the firm's real product is the quality of the judgment that produced the slate. A firm that can't show you how it defined the role and who it ruled out is asking you to absorb that failure rate on trust.

Which placement history is truly relevant?

Relevant placement history matches your role on six dimensions at once. Function and seniority matter along with market segment and company stage, and so do the commercial problem being solved and geography. A firm that placed a VP of Sales at a $2 billion device manufacturer hasn't demonstrated it can staff a first commercial leader at a company approaching its first reimbursement decision. Those are different markets with different candidates.

Push past the logo wall. Ask for the last five searches that overlap your mandate on at least four of those six dimensions, with dates, and ask what happened in the two that went badly.

Data from the Executive Search Information Exchange, cited by Cowen Partners, indicates that around 40% of executive searches never place a candidate at all. That figure is the one most firms won't volunteer, and it's why completion history matters more than placement history. A firm with a long client list and a quiet record of abandoned mandates has been paid for searches it didn't finish. Ask directly how many engagements in the past two years closed without a hire.

Comparable placements matter most

Comparable means the commercial outcome matched. Market creation for a novel device and scaling from founder-led selling to a territory model demand different leaders. Winning coverage decisions and rebuilding a go-to-market motion after a failed launch also demand different leaders.

One multi-role commercial search run by JRG Partners involved assessing more than 450 medical device sales and marketing leaders to fill a VP Sales role alongside regional positions. The useful detail there is that the firm could state the headcount.

So make the size of the assessed pool part of your request. A firm that knows how many executives it evaluated to produce a shortlist has research records. A firm that only remembers who it placed was working from a contact list.

Client references must verify performance

References are worth something only when you choose them and structure the conversation. Curated testimonials carry almost no signal, which mirrors what selection research found about reference checks generally: in the revised meta-analytic estimates published by Paul Sackett and colleagues in 2022, reference checks sit at a validity of roughly .13 for job performance, near the bottom of every method examined.

The lesson transfers directly. An unstructured reference call produces a pleasant story and nothing you can compare.

Ask instead for two clients from searches resembling yours, and ask each about shortlist quality against the original brief and how the placed executive performed against first-year goals. Retention past twelve months is the number that most honestly reflects the assessment work.

Can the firm reach your market?

Reach means the firm can identify and get a reply from qualified people who aren't looking, across your whole target segment and among people who have never appeared in its database. Database depth measures past activity. Your search needs present-tense access.

LinkedIn data reported by Alpha Apex Group puts 70% of the global workforce in the passive category, and at commercial leadership level in a narrow device segment the proportion runs higher still. The people who can take your product from pilot accounts to a repeatable revenue motion are employed and compensated well.

Which means the firm's database is the wrong thing to be impressed by. A candidate already in the system is, by definition, someone who has been approached before, and in a segment of 300 relevant executives, recycled contacts get you a shortlist drawn from the same dozen names every competitor has already called. Ask what percentage of the last comparable shortlist came from new research rather than existing records.

Market maps reveal real expertise

A sample market map is the fastest test of whether a firm understands your segment. Firms doing genuine research routinely identify and rank more than 100 executives per search before any outreach begins, according to industry process documentation compiled by Consultadd.

Ask for a redacted map from a prior search and check whether it contains:

• Named target companies grouped by why they're relevant, not by size

• Candidate archetypes tied to the commercial problem, with the trade-offs of each

• Adjacent pools worth testing, such as diagnostics or specialty pharma, and the conversion risk of each

• Geographic and compensation constraints stated as numbers

A map with no adjacent pools tells you the firm plans to recruit from your direct competitors only. That narrows your options and raises the odds the person you hire arrives with a non-compete problem.

Access claims need qualification

Access claims need boundaries attached, and the boundary that matters most is off-limits. Every client a firm signs removes an employer from the pool it can recruit from for a period after placement.

Caldwell Partners describes the arithmetic plainly: a firm running five concurrent CFO searches has roughly 25 or so executives already committed to other slates and unavailable to the next client. Scale that across a large firm's active roster in a concentrated sector and a meaningful share of your market disappears before work starts.

Ask for the off-limits list in writing, with expiry dates, before you sign. Ask for expected response rates from comparable outreach and how many searches the assigned team is running concurrently.

Is the search process operationally credible?

A credible process converts your mandate into a written campaign with stages and dates you can hold someone to. The test is whether the proposal describes what the firm will do for you or what the firm does generally.

Benchmark data from the Clockwork Recruiting report puts the average retained search at 123 days from kickoff to accepted offer. Any proposed schedule materially shorter than that needs an explanation grounded in something real, like an existing mapped pool in your segment.

Here's the inference worth drawing. Since the average sits at four months, a firm promising a shortlist in three weeks is either presenting candidates it already had on hand for another client, or it's planning to compress the research phase that produces non-obvious names. Both routes deliver a faster slate and a narrower one. Ask which weeks of the 123 the firm intends to remove, and what it gives up by removing them.

Methodology should show every stage

Methodology is credible when every stage has a deliverable attached to it. Role definition and research need to appear with a named output, and so does every later stage through the follow-up after the start date.

Your own time commitment is part of the plan. KORE1, a retained search firm, estimates that a full executive search consumes 20 to 30 hours of client executive time, front-loaded into a kickoff of four to six hours and clustered again around interviews.

A firm that hasn't told you that is either planning to run without your input or hasn't thought about your calendar. Both are problems. Ask for the hours expected from you by week, because a search that stalls waiting on your availability still burns the clock you're paying for.

Senior involvement must be explicit

Get the names of the people doing the work into the contract. Five jobs make up the work, and at larger firms they're rarely done by the person in the pitch meeting.

Hager Executive Search notes that at large global firms, searches are run by associates after the initial partner conversation, with longer timelines and broader off-limits restrictions than growth-stage companies expect.

The practical consequence is that your calibration conversations, the ones where you refine what "good" looks like after seeing the first three profiles, happen with someone who wasn't in the room when the mandate was set. Ask for the partner's committed hours per week and what happens if the first shortlist misses. Written escalation beats goodwill.

Timelines need assumptions and milestones

A timeline without stated assumptions is a guess with dates on it. Require the dependencies in writing: your interview availability and board sign-off windows, along with holiday periods and how long notice runs in your market.

Published averages won't help you here without adjustment. SHRM's State of Recruiting 2025 put median time-to-fill for executive positions at roughly a month and a half, a figure covering all executive hiring rather than confidential retained searches in scarce segments.

The gap between that median and the retained benchmark cited earlier is the space where firms set expectations they can't meet. Ask which figure the proposed schedule is built from. Then ask how confidentiality holds during outreach, specifically at what point your company name is disclosed and who inside the firm can see the assignment file.

How rigorous is candidate evaluation?

Rigorous evaluation produces comparable evidence across candidates, collected the same way, scored before anyone discusses it. Résumé matching and conversational interviews produce impressions that feel decisive and predict little.

The evidence on this is settled. In the 2022 Journal of Applied Psychology meta-analysis by Sackett and Zhang, along with Berry and Lievens, structured interviews reached a mean validity of .42 versus .19 for unstructured ones. Structure was the highest-validity single method they examined. Those are correlations with later performance.

Apply that to what a search firm hands you. If the deliverable is a written summary of a free-flowing conversation, you're being given the .19 method dressed as insight. Ask whether questions were set before the first candidate call and whether ratings were recorded independently.

Assessments should match role outcomes

Assessment content has to come from the job. The competencies should read like your commercialization plan translated into behaviors.

Structure alone isn't enough, though. Sackett and colleagues reported an 80% credibility interval for structured interviews running from roughly .18 to .66, a spread wide enough to cover both excellent and near-useless implementations.

That range is the whole argument for inspecting the instrument rather than accepting the label. Ask to see the competency framework built for your role and the anchored scoring scale. Ask for the reference protocol and one anonymized example of how the firm documented conflicting evidence about a candidate. How disagreement gets resolved tells you more than how agreement gets recorded.

Diversity requires a sourcing method

Broadening the pool is a research task, and it belongs at the sourcing stage where the criteria stay identical for everyone. Research summarized by the Harvard Kennedy School People Lab found that finalist pool composition itself shifts evaluation, with the odds of selection rising sharply once at least two candidates from an underrepresented group appear rather than one.

The mechanism is about how evaluators read an outlier, which means the intervention that works is on the research side.

So ask the firm how it widens the target list: which adjacent segments and company types it adds, and how it tracks pool composition through each stage. Also ask what it reports when the qualified pool is genuinely thin. A firm that says the market constraint out loud is more useful than one that promises a balanced slate and quietly relaxes the role requirements to produce it.

How should firms be compared?

Compare firms with one weighted scorecard filled in from evidence. Weight the criteria before the first pitch, while nobody has charmed you yet.

There's real money behind the discipline. SHRM's 2025 benchmarking data puts average recruitment cost at $35,879 for executive hires, and that figure excludes the retained fee and the months of unclosed revenue.

Score each firm from 1 to 5 on role relevance and market access. Apply the same scale to methodology and assessment rigor, then to sourcing breadth and named senior involvement. Reference verification and timeline realism get scored too, as do confidentiality controls. Weight role relevance and market access highest for a specialized mandate, because those two can't be fixed mid-search. Methodology can be tightened after kickoff. A firm blocked out of half your segment by off-limits agreements cannot be.

Ask every reference identical questions

Use the same question set with every reference, in the same order, and write down answers before you form a view. Varying the questions makes the calls incomparable, which is the same flaw that drags unstructured candidate interviews down.

Expect guardedness. As Frank Schmidt and In-Sue Oh noted in their review of a century of selection research, employers remain reluctant to provide more than dates of employment and job titles when giving references.

Reference-givers for search firms are friendlier but similarly cautious, so ask questions that require a number or a story instead of a judgment. Cover what was promised versus delivered and shortlist quality against the brief. Ask about communication during the slow weeks and how the firm handled a setback. Ask whether the placement lasted and whether they'd hire the firm again for a different role.

Investigate evidence gaps before selection

Treat every gap as unresolved until the firm closes it in writing. Vagueness at the proposal stage becomes surprise at week ten.

The gaps worth chasing:

• Examples described by industry rather than by role, outcome, and date

• Guaranteed speed claims, which sit oddly against the 120-day executive average reported in 2026 time-to-hire data compiled by The Resource Company

• Sourcing plans that lean on database size rather than fresh research

• Off-limits lists that arrive after contract signature

• Assessment described as judgment or instinct with no scoring artifact

• References from searches unrelated to your function or stage

A firm quoting a time-to-fill well under that average for a specialized commercial role is describing a different kind of work. Make it explain the difference before you weigh the number as a strength.

Turn findings into a final matrix

Put everything into one document before you decide, because the comparison falls apart the moment it lives in separate notes and memories. Build a matrix with firms as columns and your weighted criteria as rows, then fill each cell with the specific evidence you verified rather than a general impression. Add two rows at the bottom: unresolved questions, and the risk you'd be accepting if you chose that firm anyway.

The matrix does something a meeting can't. It shows you which firm performed well on the criteria that matter most for your mandate and which strengths look impressive but have little bearing on the search you're actually running.

TruAlign Partners specializes in executive search and commercial recruiting for growth-stage and commercial-stage medical device and MedTech companies. Its CarbonCore process starts by defining what success in the role should look like, then evaluates candidates across three dimensions: how they present in the interview, what their experience and qualifications show they can do, and how they are likely to approach and perform in the specific role and environment. Behavioral insights complement, rather than replace, experience, interviews, references, and professional judgment.

If TruAlign is on your shortlist, ask the firm to document its search process and assessment framework against the same criteria you're applying to every other firm, then evaluate the evidence the same way.