Photo: free stock photography (Unsplash licence) — see imprint
Chunking decides everything
Split a datasheet in the middle of a table and every answer built on it will be wrong. Chunk along document structure — headings, table rows, sections — not by a blind character count. This unglamorous step determines most of your final quality.
Retrieval, then generation
If the right chunk is not retrieved, the best model on earth will confidently make something up. Measure retrieval separately: for 50 real questions, is the correct passage in the top five? Fix that before you touch prompts.
Always cite, always allow 'no'
Every answer should link the source document, and the system must be allowed to say it does not know. An assistant that invents a torque value is worse than no assistant — someone will tighten a bolt to it.
Keep your data in your house
Quotes, prices and customer names are not training material for someone else's model. Use an endpoint with a no-training guarantee, or self-host — and write down which data leaves the building. Your DPO will ask.
- Chunk along structure, never by character count.
- Measure retrieval before you tune prompts.
- An assistant that cannot say 'I don't know' is dangerous.
Frequently asked questions
The model calls are the smallest line on the invoice, and people are always surprised by that. The work is in your documents: chunking them along their structure, building an evaluation set of real questions, and measuring whether retrieval actually finds the right passage. If your datasheets are clean PDFs with real headings, that shortens. If they are scans, it does not.
Almost always retrieval, not the model. If the right passage never reaches the model, the model fills the gap confidently — that is what it does. Test the two halves separately: take fifty real questions and check whether the correct passage lands in the top five results. Fix that number before you touch a single prompt.
That depends entirely on the endpoint you use, and it is a question you should answer before the project starts, not after. Either use a provider contract with a no-training commitment or host the model yourself. Either way, write down which categories of data leave the building and where they go. Your data protection officer will ask, and a shrug is not an answer.
That is the demo, not the system. Tools that split documents by character count will happily cut a datasheet through the middle of a table, and every answer built on that half-table is wrong in a way nobody notices. Splitting along headings, sections and table rows is unglamorous and it decides most of your final quality.
We do this for a living — Shopware, Node.js, React, ERP integration and automation for B2B.
Talk to an engineer