SC

Strategy Canon

Strategy Map›Layer 4 · Product Strategy›Product Quality+ from outside the named canon

Accuracy

By Dave WhittemoreUpdated

What is Accuracy?

Accuracy, as a product strategy, means customers choose a product because its core output is right more often than a rival's: a security tool that catches more threats, a test that reads more samples correctly, an estimate closer to the real price. The gap has to be measured in a trial, benchmark or independent test.

Also calledoutput qualitycorrectnessevals (Husain and Shankar)

How Accuracy works

Where correctness is the reason to buy, a measured lead gives buyers something concrete to act on. It can win a head-to-head evaluation, persuade an insurer to pay for a test, or let users stop double-checking the output.

Accuracy decides markets less often than it seems, because buyers also weigh speed, price, workflow and switching costs. In 2017 an NSS Labs test scored Cylance above CrowdStrike on detection, and CrowdStrike still won the market. A trial showing a product beats usual care is evidence that it works. It says nothing about how the product compares with a rival.

A checkable answerA detection, diagnosis, estimate or match
Measured against a rivalSame cases, same test, both products scored
The lead winsBuyers choose it, payers reimburse it
Other factors winCylance outscored CrowdStrike; CrowdStrike won anyway
What a measured accuracy lead can lead to: sometimes the buyer's choice, sometimes nothing, as in Cylance's 2017 detection win over CrowdStrike.

Accuracy vs. Reliability and Speed

What it isHow to tell it apartExample
AccuracyCustomers choose it because its core output is correct more often than a rival'sLook for a measured comparison of right-or-wrong output against a named rivalNSS Labs scoring Cylance against CrowdStrike on detection
ReliabilityThe product fails, crashes or drops work less often than the alternativeAsks whether it worked every time; a working product can still give wrong answersTeams leaving HipChat for Slack after failed message delivery
SpeedFaster than the alternative at the customer's core taskMeasured in time to finish the task; the result's correctness is a separate questionReviewers finding Spotify faster and more responsive than Rdio

How to tell if a company has it

  1. An output that can be wrongIts correctness is the point of the product: a detection, test result, estimate, match or prediction.
  2. A measured comparison with a named rivalA trial, benchmark, independent test or formal evaluation scoring both products on the same cases. A comparison with usual care doesn't count.
  3. The difference changed a decisionThe lead shows up in adoption, reimbursement, a buyer's choice or a head-to-head evaluation the company won.
  4. Same-period tests point the same wayIndependent tests from the same years don't favor the rival. Cylance led one detection test and still lost the market.

Examples

  • NSS Labs' 2017 test scored Cylance above CrowdStrike on detection. CrowdStrike still won the market.
  • Two listing-accuracy studies found Zillow and Trulia about even.

Who names it

AuthorWhat they call it / where it appears
Husain & Shankar +Evals: measuring whether an AI product's output is correct
← ReliabilityEase of Use →