A Python pipeline that turns full-length surgical-textbook PDFs into clean, structured, machine-readable data: 220 chapters into a searchable knowledge base of sections, figures and tables, exported as a versioned, hash-verified package the search platform loads directly into its graph and vector databases.
sushrutalgs.ai answers surgical-exam questions and traces every claim back to a line in a standard textbook, so a citation is only worth anything if the text behind it is intact and addressable down to the section, figure or table. What actually existed was three full surgical textbooks as print PDFs: 220 chapters and 5,941 pages laid out for a typesetter, where a table is drawn rather than encoded and a figure's caption is a floating run of text that happens to sit near it. None of the retrieval work could start until that was structured data, and nobody was going to re-key three textbooks by hand.
PDF extraction fails silently. A parser that loses content does not raise an error, it returns a slightly smaller document, and every downstream count still looks plausible, so the loss only surfaces months later when someone asks about a topic the book plainly covers and the answer has nothing to cite. One bug in the extraction path quietly dropped 968 table elements while the run reported success. The second half of the problem is that layout rules tuned to one publisher break on the next, so a single static rule set cannot be trusted across three books, and "it ran without errors" is not evidence of anything.
Two rules shaped the pipeline: structure is decided only by code that behaves the same way twice, and every stage has to be able to say what it lost.
All 220 chapters and 5,941 pages went through end to end, and all 220 exported chapter packages pass structural validation. The check that mattered most was a clean rebuild from the source PDFs reproducing every count, because that is the only way to know the output does not depend on incremental state left over from earlier runs. The honest limit is that the validation is structural: it shows nothing went missing or landed in the wrong chapter, but a model-written figure description is still a model's description, and those are spot-checked by hand rather than measured. A fourth textbook would still need parser work; the pipeline adapts, it is not publisher-agnostic.
In numbers
The pipeline's job was never speed, it was trust: 5,941 pages a citation product has to be able to stand behind. A parser that fails loudly instead of returning a slightly smaller document, and an export that is hash-verified end to end, is what makes that trust something other than a hope.