Vendor Benchmarking: Comparing Contractors Fairly
Vendor benchmarking only works when scope, standard and scoring method are identical. Here is how to compare contractors without producing misleading numbers.
Vendor benchmarking gives you two numbers you did not have before: how a contractor performs against the standard, and how that performance compares with other buildings, other contractors or the same contractor a year ago. Both are useful. Both are also easy to get wrong, because a benchmark is only as good as the comparability behind it. Two contractors working to different scopes, measured by different people on different scales, produce a comparison that looks authoritative and means nothing.
This is the discipline that turns inspection records into management information, and it is the analytical arm of property management vendor management.
Key takeaways
- Normalise scope before you compare scores. Same task list, same frequency, same area types across every vendor being benchmarked.
- One scoring scale, used everywhere. APPA 1–5 applied the same way to every site makes scores comparable; percentage-complete or pass/fail does not.
- Sample the same area types at the same frequency. A washroom-heavy building will always score worse than one dominated by corridors if you compare raw averages.
- Compare trends within a vendor before comparing vendors. A stable 4 beats a volatile 3.6 every time.
- Small samples mislead. Six inspections a quarter at one site is a reasonable base; one inspection is an anecdote.
- Publish the method with the result. A benchmark the vendor cannot audit will be disputed, and usually should be.
What vendor benchmarking actually measures
Benchmarking answers one of three questions, and it is worth being explicit about which one you are asking.
Absolute performance. How well is this contractor meeting the published standard right now? Measured against the specification, not against another vendor.
Relative performance. How does this vendor compare with another vendor delivering similar scope in a similar building? Requires normalised scope.
Longitudinal performance. How does this vendor compare with its own performance last quarter or last year? Requires a stable scoring method over time, and it is usually the most actionable of the three.
The third is where most managers get their return. A vendor moving from 3.2 to 4.1 over three quarters tells you the improvement plan worked. A vendor drifting from 4.3 to 3.6 tells you to intervene before renewal.
What must be identical before any comparison is fair
Four things have to match. If any one of them does not, the benchmark is invalid.
| Element | Why it must match | What to check |
|---|---|---|
| Scope and task list | Different tasks produce different scores | Same checklist items per area type |
| Frequency | Five services a week scores better than three for the same effort | Frequency stated in both contracts |
| Area type mix | Washrooms and entrances score lower than offices and corridors | Weighted average by area type, not raw |
| Scoring method and inspector competence | Two people scoring the same corridor differently destroys comparability | Same scale, calibrated inspectors |
The fourth is the one that quietly ruins benchmarking programmes. If the manager scores entrances on Monday and the supervisor scores them on Friday, the difference between those two scores may be entirely about who held the clipboard. Calibration sessions — two inspectors scoring the same area independently and reconciling the difference — are the only reliable fix, and the approach mirrors the calibration discipline used in contractor-side inspection scoring calibration.
Standardise the scale before you standardise anything else
The APPA five-level cleanliness scale is the de facto vocabulary for commercial cleaning quality in North America, and it is used precisely because it allows a manager, a vendor and an owner to mean the same thing when they say "a 3". Using it consistently across every vendor gives you a benchmarkable series.
If you want evidence that a contractor runs a documented management system rather than an informal one, the ISSA Cleaning Industry Management Standard is the reference most building service contractors in North America are assessed against, and certification status is a reasonable pre-qualification question.
A workable scoring convention:
- 5 — pristine, exhibition standard, no visible soil
- 4 — clean and well maintained, minor isolated issues
- 3 — acceptable, standard met with noticeable deficiencies
- 2 — below standard, visible soil in multiple locations
- 1 — unacceptable, requires immediate corrective action
Two details matter for comparability. First, weight items by consequence rather than treating every line equally, so a soiled washroom does not average away against a clean corridor. Second, flag critical items — hygiene, safety, blockage and odour — separately, because a contractor with three critical failures should not show the same score as one with three dusty ledges — critical. A single missed washroom servicing or a blocked sanitary bin is a different class of event from a smudged glass panel, and averaging the two together flatters the wrong vendor.
Weighted averages by area type: the core technique
Raw averages across buildings are the most common benchmarking error. Consider two sites:
| Site | Office areas (score) | Washrooms (score) | Entrances (score) | Raw average | Weighted average |
|---|---|---|---|---|---|
| Site 1 — 70% office, 15% washroom, 15% entrance | 4.4 | 3.6 | 3.9 | 3.97 | 4.23 |
| Site 2 — 40% office, 40% washroom, 20% entrance | 4.4 | 3.6 | 3.9 | 3.97 | 4.11 |
Both sites have identical area scores and identical raw averages, yet the second site is doing more of the harder work. If you benchmark raw averages across a portfolio, you will conclude both vendors perform equally and you will learn nothing about which building is harder to hold to standard.
Weight by the area mix each vendor actually services, and benchmark area type against area type. The comparison that carries information is "washrooms at Site 1 versus washrooms at Site 2", not "Site 1 versus Site 2".
Cadence, sample size and what to report
A benchmark needs enough observations to be stable and few enough to be affordable.
- Monthly inspection of a rotating sample covering every area type at least once per quarter.
- Minimum six inspections per vendor per quarter before publishing a comparison. Below that, report the findings and withhold the ranking.
- Quarterly trend, showing the current quarter against the previous three.
- Annual review with a three-year series where records allow.
Report four numbers per vendor: weighted score, critical-item count, deficiency closure time, and repeat-finding rate. Those four together resist manipulation better than any single score. Closure time and repeat findings are the two that expose a vendor who cleans for the inspection rather than for the building.
A benchmark table you can use in a review meeting
| Vendor | Sites | Weighted score (Q) | Prior quarter | Critical items | Median closure | Repeat findings |
|---|---|---|---|---|---|---|
| Vendor A | 3 | 4.1 | 3.9 | 0 | 1 day | 8% |
| Vendor B | 2 | 3.6 | 3.7 | 2 | 4 days | 22% |
| Vendor C | 1 | 4.4 | 4.4 | 0 | Same shift | 4% |
The story is in the columns, not the headline score. Vendor B is not far behind on score but is materially behind on critical items, closure time and repeat findings. That is the conversation the review meeting should have, and it is also why score alone should never be the only published metric. The meeting format that works for these discussions is described in running a quarterly vendor performance review.
Common benchmarking mistakes that produce false conclusions
Comparing across different scopes. A vendor cleaning five days a week will outperform a three-day vendor on the same checklist. Normalise frequency first or the result is predetermined.
Changing the checklist mid-year. Every item added or removed breaks the series. Version the checklist, date the version, and note when it changed.
Letting the vendor self-score. Self-assessments have a place — they build ownership in the crew — but they cannot be the benchmark. Independent inspection is the number you publish.
Publishing a ranking with no method note. State the scale, the sample size, the inspection dates and who inspected. A benchmark nobody can audit will be dismissed, correctly, as opinion.
Benchmarking only to rank. Ranking is the least useful output. The useful outputs are the area types where everyone underperforms, the practices that separate the best performer from the rest, and the early warning that a vendor is drifting.
What to do when the numbers come in
Benchmarking produces three findings, and each has a different response.
One vendor is clearly behind. Start with a documented improvement conversation, set dated targets against specific area types, and re-inspect the same areas within 30 days. If two consecutive re-inspections do not move the score, escalate formally.
Everyone is behind in the same area. That is usually a specification problem, not a vendor problem. If every contractor scores 3 on stairwells, the frequency or the task list for stairwells is wrong, and you should rewrite the cleaning specification rather than replace a contractor.
One vendor is clearly ahead. Find out why and write it down. Ask what frequency, equipment, staffing ratio and supervision model they use, then take that finding into your next tender as the standard you expect others to meet.
Where benchmarking stops being useful
Benchmarking is a diagnostic tool, not a management system. Two limits are worth respecting.
First, benchmarks measure delivery against your specification, not whether the specification is right. A vendor can score 4.5 on a specification that under-serves the building.
Second, comparisons across genuinely different building types have a short shelf life. A childcare centre, a warehouse and a Class A office share a scale but not a workload. Benchmark within a peer group, and state the peer group when you report.
Scope note. Where benchmarking reveals a hygiene, air quality, water or building-system problem — mould, ventilation, water intrusion, pest activity — that is outside a cleaning inspection's remit to diagnose. Record and photograph what you observe, report it to the owner, and commission the appropriate qualified contractor rather than directing the cleaning vendor to treat it.
Frequently asked questions
How many vendors do I need to benchmark?
Two is enough for a relative comparison, though three or more makes the peer group more meaningful. If you have one vendor, benchmark it against its own history and against the specification — that is still valuable and often more actionable than a cross-vendor ranking.
Can I benchmark vendors across different building types?
Only within a peer group. Compare offices with offices, retail with retail, and clinical space with clinical space. A cross-type comparison using raw averages will mainly measure the difficulty of the buildings, not the quality of the contractors.
How often should I publish benchmark results?
Quarterly for internal management, annually for owners and boards. Monthly reporting produces noise; annual-only reporting produces surprises. Quarterly gives the vendor time to act between reports and gives you enough data points to see a trend.
What if a vendor disputes the scores?
Show the photographs and the item-level scoring. A rating a vendor can trace to a specific location, a specific item and a dated photo is very hard to argue with. Where the dispute persists, offer a joint inspection of the same area and reconcile on the spot.
Should benchmarking results affect payment?
They can, if the contract says so — a service credit tied to critical items or to closure times is enforceable. Do not introduce payment consequences retroactively. Put the mechanism in the contract at award or at renewal, with the scoring method described in enough detail that both sides know how it works.
Where NeatScore fits into benchmarking
Benchmarking is only possible when every vendor is measured the same way. NeatScore is a cleaning inspection tool built for exactly that: identical checklists across sites, APPA 1–5 scoring with weighted and critical items, photos attached per item, offline inspections that sync, and dashboards that put sites side by side without a spreadsheet rebuild.
It gives you a comparable series from the first month, which is the only way a benchmark becomes meaningful by the second quarter. Try it free for 14 days, or see pricing.
