A vendor tells you their OCR is 99% accurate, and the number does its job: you stop asking. I spent years in the 2000s building statistical models to correct OCR output on recorded documents, and the thing I learned first is that character accuracy is the least informative number in the pipeline. A plant can have 99% character accuracy and a 15% error rate on the fields an examiner actually searches, and both numbers can be true at the same time. The September 3 post covered what happens to the image before any software sees it. This post is about the layers above the image, because that is where a good scan still turns into a bad index.

Five Layers Between the Page and the Record

Start with the pipeline as it actually runs. First, image to characters: the OCR step, the one with the 99% on the brochure. Second, characters to structure: deciding which marks form words, which words form lines, where a column breaks, where a legal description wraps onto the next page, which block of text is a stamp and which is the body. Third, structure to fields: deciding that this name is the grantor and that one the grantee, that these digits are a book and page and those are an instrument number, that this paragraph is the legal description and that one is the consideration clause. Fourth, fields to normalized values: “Smith, John A.” and “JOHN A SMITH” and “John Smith et ux” resolved to something a search will find; a legal description parsed into lot, block, and subdivision, or section, township, and range. Fifth, values to records: linking the instrument to a parcel, the parcel to a chain, the chain to a file.

OCR is layer one. Extraction is layers two through four. Indexing is layer five. When a vendor quotes you one number, ask which layer it describes, because almost always it is the first, and the first is the one your examiners never look at.

Why the Errors Compound

Character accuracy sounds high because it is measured per character, and nobody searches by character. An instrument number is twelve characters. At 99% per-character accuracy, the probability that all twelve are right is 0.99 to the twelfth power, which is about 89%. One instrument number in nine is wrong, in a plant that is “99% accurate.” A forty-word legal description at the same rate is right, end to end, less than a third of the time. The number that mattered was never the character rate. It was the field rate, and the field rate is what the character rate looks like after you multiply it by the length of the things you actually need.

Then notice where the errors live. They are not spread evenly across the page. They cluster in handwriting, in stamps, in the margins, in the 1974 typewriter with the worn lowercase e, in the county that photocopied its microfilm. Those are also, disproportionately, where the recording information and the names live. A pipeline can be 99.5% accurate on the typed body of a warranty deed and 80% accurate on the clerk’s stamp, and the stamp is the part the index is built from.

Confidence Scores Measure the Wrong Thing

Every OCR engine returns a confidence score, and every extraction pipeline I have seen misuses it. OCR confidence is per character and answers one question: how sure is the engine that this mark is this letter? It has no idea whether the field is right. “Smith” can be recognized with perfect confidence and assigned to the wrong role with the same confidence, because role assignment happens two layers up, where the OCR engine’s opinion doesn’t reach. A pipeline that routes human review by OCR confidence sends your examiners to look at the blurry characters and waves through the clean ones that were put in the wrong column. The review queue fills with the errors that don’t matter and empties of the ones that do.

Field-level confidence is a different thing, and it has to be built. Does this grantor name match a party on the prior instrument in the chain? Does this book and page fall in the sequence the county was recording that week? Does this legal description resolve to a parcel that exists? Those checks live in layers four and five, and they catch the errors that character confidence is blind to.

Where the Language Models Fit

The newer extraction models are good at layers two through four. They read layout the way a person does, they know a grantor clause when they see one, and they can normalize a name without a rulebook. That is real progress and we use it. But two things haven’t changed. They still read whatever layer one hands them, so an unreadable stamp is an unreadable stamp. And they are confident in the same way OCR is confident: fluently, and without reference to the record. A language model will produce a plausible book and page from a smudge and present it as clean. The July 7 post called that confident garbage, and the only defense against it is the same as it was in 2006: check the extracted value against something outside the document.

The Only Number Worth Asking For

So stop asking for OCR accuracy and ask for field accuracy, measured on your documents. The test fits in an afternoon. Pull a hundred instruments from your plant, stratified by decade, because the 1960s and the 2010s are different problems. For each one, compare the index record to the image, field by field: grantor, grantee, instrument type, book and page or instrument number, legal description, recording date. Score each field right or wrong. Then look at the table by field and by era rather than at the overall average, because the average will hide a 30% legal-description error rate in pre-1980 instruments behind a 98% rate on everything recorded last year.

That table is the number. It tells you where your searches are failing, which eras to re-index, and what any vendor claiming to improve your plant has to beat. A vendor who will only quote character accuracy has either not measured field accuracy or would rather you didn’t.

How We Built Autopilot Around This

Autopilot was designed by people who had already lost the argument with character accuracy. Every field it extracts is validated against something outside the page: names against the parties on adjacent instruments in the chain, recording information against the county’s sequence, legal descriptions against the parcel they claim to describe. Review is routed by field confidence, not character confidence, so the examiner’s time goes to the grantor that doesn’t match the chain rather than the smudged date that does. And the accuracy we report is measured the way this post describes, by field and by era, on your documents. OCR is where the pipeline starts. It is not where the index comes from, and it is not the number you should be buying on.

See how Autopilot validates every field, not just every character →