With RAG, an AI system can generate responses to user queries by referring to information stored in external documents. The process of creating a robust RAG system is hindered when knowledge bases include scanned PDFs, tables, financial reports, or documents with many images.
For instance, you may ask the AI assistant: "What was the company's revenue in Q3?" The information is in the financial report, but the system provides you with the wrong number. The error may not result from a language model – the PDF may have been extracted incorrectly, the table structure has been lost, or the retrieval system has chosen an irrelevant chunk of the document.
All the challenges show why RAG document processing is critical for creating an accurate AI application. Good performance requires not only embeddings and a vector database but also proper information extraction, document structure preservation, appropriate RAG chunking techniques, and correct retrieval of relevant evidence.
Why RAG Systems Have Difficulties with PDFs
PDF documents are designed to maintain visual layout, not to be convenient for AI systems to process. A document can be readable for a person but is hard for a RAG system to process due to text, tables, and images.
There are three main reasons why a RAG pipeline may be inefficient.
1. Scanned PDF Contains Images Rather than Text
Scanned PDFs contain photos of printed pages, rather than selectable text. Standard tools for text extraction may return minimal content from these files.
OCR converts images of texts into a machine-readable text format. Without OCR, a RAG system will not be able to index information for answering the question.
2. Table Structure Is Lost
Imagine a financial report that contains revenue figures for various quarters. During extraction, these figures may get separated from the table row names or from reporting periods.
The system retrieves the revenue figure without recognising which quarter it is for. Even when the correct number is indexed, the lack of context results in the incorrect answer.
3. Document Structure Is Lost
The information relating to the document’s structure, like the heading, footnote, page number, table caption, etc., is essential. If it gets lost during document processing, the retrieval system may provide the wrong evidence.
The solution is in developing a document processing pipeline based on the real document structure.
How to Improve RAG Document Processing with PDF OCR
The first step in reliable document processing is in understanding what kind of document is entering the pipeline.
For text-based PDFs, the extraction tool can retrieve existing text. Scanned documents require OCR. Documents with charts, tables, complicated layouts, etc. may need image and layout analysis in addition to OCR.
Practical document processing includes these steps:
-
Document Type Identification: Determine if the document contains selectable text or is a combination of scanned pages containing tables and images.
-
Content Extraction: Utilise an appropriate PDF parser when dealing with textual PDFs and OCR when dealing with scanned pages.
-
Structure Preservation: Preserve the structure of the document, including headings, sections, page numbers, and so on.
-
Text Cleaning: Cleaning of extracted text in order to correct common errors, removing unnecessary data and inspection of suspicious text and data.
-
Output Validation: Validation of extracted text and tables in comparison with the original PDF document.
How to Make PDF OCR Readable for RAG
The output of OCR is not necessarily accurate. Bad scan quality, unusual font, low-resolution images, complicated layout and other factors can influence the quality of OCR output.
The ways to improve OCR quality are in providing sufficient clear source images, correcting page orientation, applying pre-processing to images when needed and validating important numbers and identifiers. Preserving of page references allows tracing back answers to the original document.
For example, image-to-PDF text recognition makes a scanned invoice searchable. However, bad OCR output can misread an invoice amount. Therefore, the RAG system can return the wrong value. It is important to pay particular attention to date, account number and other pieces of information.
In addition to making the document searchable, we also need to make sure that the extracted text is correct.
Choosing the Correct Method of RAG Chunks
After text extraction, the next stage is text chunking for indexing and retrieval.
The chunks may be too large and consist of unnecessary information. On the other hand, chunks can be too small and lack some essential information for the interpretation of the answer. The proper chunking techniques of RAG ensure the right balance between retrieval precision and contextualisation.
- Fixed Size Chunks
The fixed-size chunking technique splits text into equal chunks based on the given token or character limit, and there is an overlap of neighbouring chunks.
The algorithm is easy to implement and effective in the case of relatively even text. However, such chunking may break up paragraphs, separate headings from their descriptions and split tables from column titles.
- Structure-Aware Chunking
Structure-aware chunking takes into account the structure of a document (headings, paragraphs, sections, tables, etc.).
This kind of chunking can be particularly valuable for business documents, as it preserves the connection between a heading and its contents. Moreover, such a chunking technique simplifies the metadata addition process for the chunks (e.g., document name, page number, and section).
- Parent-Child Chunking
Parent-child chunking ensures the creation of smaller chunks for better precision but also allows keeping a larger chunk for the context.
For instance, the small chunk that contains a financial figure may correspond to the user query. Then the system can retrieve the bigger chunk to learn the reporting period, other metrics and explanations.
Best Practice: do not select the chunk size according to the universal rule. Instead, test different configurations against sample questions from your own documents and measure if the correct evidence is retrieved.
Handling of tables using RAG with multimodality
Among the most difficult to handle objects are tables because of dependencies between the rows, columns, the header, and values contained there.
A table extraction process should not break these relations. For instance, a table containing quarterly revenue could be transformed to
Metric: Revenue
Reporting period: Q3
Value: ₹25 crore
The structured representation provides additional information to the retrieval system and allows distinguishing similar values from the same reporting period.
Simple tables could be converted to contextual text. For complex financial and scientific tables, it is better to use structured storage. The choice depends on the way users are going to query the information.
When is multimodal RAG helpful?
Thus, the multimodal RAG extends the scope of searching for information up to non-textual information (images, graphics, tables, etc.).
Let us consider an example where the technical report contains the graph that is absent in the text. In such a case, the textual-only method could miss the information. Multimodal RAG allows combining the information from the graph and extracted text and thus increases the accessibility of the information.
It should be stressed that multimodal RAG does not replace the OCR or the extraction of the structured table technique. They solve completely different problems, and the usage of them together should be determined by the documents' content and requirements of the application.
Improving retrieval by means of metadata filtering and exact match
Even the optimal extracted information could lead to the wrong answer if the retrieval system selects the wrong passage.
Metadata filters allow limiting the search space, taking into account the characteristics of the document (type, period of reporting, department, client, page number, etc.). For example, if a person needs to get Q3 revenue information, then there is no need to take information from Q2 just because there is the information about the revenue in both texts.
An exact match can be used if there are unique identifiers (invoice numbers, product codes, dates, and financial figures). Semantic search can be applied to find similar concepts.
A robust RAG pipeline could use the following steps:
- Filtering using the metadata to limit the search to the relevant documents or period.
- Keyword and exact match search to find exact terms, identifiers, and values.
- Vector search to find semantically similar passages.
- Re-ranking to sort the candidate passages by relevance.
Using LangChain for the implementation of the RAG pipeline, these methods could be combined into the workflow that retrieves document chunks, feeds the relevant context to the language model, and generates a grounded answer. The components would depend on the choice of models, vector store, and document processing methods.
How to test whether your RAG system works?
The RAG system should be tested on the representative queries and known source documents instead of the few positive examples.
It should include the queries regarding the scanned PDFs, table values, reporting periods, specific document-related terminology, and information distributed in several sections of the document.
Test the following aspects:
- The accuracy of extraction: was the source content extracted correctly?
- The quality of retrieval: did the system retrieve the passages containing the answer?
- The completeness of the context: were the relevant headings, labels of tables, and additional explanations preserved?
- The correctness of the answer: does the generated answer match the source?
- The accuracy of extraction: is the content of the source extracted correctly?
- The quality of retrieval: is the answer found by the system in the passages from the document?
- The completeness of the context: are all the relevant headings, labels of tables, and other context provided?
- The correctness of the answer: does the generated answer reflect the source?
- The accuracy of citation: is it possible to find the relevant information in the original document?
If there is an incorrect answer, then try to understand what was the cause of such result? The incorrectly extracted text means that there were problems in the document processing stage. The absence of evidence points at the problems of chunking and retrieval. Unsupported claims show the issues in the generation of the answer.
In such a way it is possible to solve the root cause of the problem instead of prompt adjustment only.
Designing the reliable RAG systems for business documents
The PDF-based RAG system implies a number of consistent decisions on the document processing, OCR, chunking, metadata extraction, retrieval, and answer generation. One of the weak links of the system could affect the whole system negatively.
Simple text documents may be processed using basic extraction and retrieval. Complex business documents require such steps as OCR, structure-aware chunking, table extraction, metadata filtering, exact match search, and multimodal retrieval.
The developers of the production-ready AI applications need to consider the format of the documents, questions types, data sensitivity, and accuracy requirements when designing the architecture.
Learn how to design a document-focused RAG-based AI application to solve business tasks.
Conclusion
The reliability of the RAG system is not only in the ability of the language model to generate an answer. It depends on such elements as accurate PDF OCR, chunking strategy, table extraction, metadata filtering, and retrieval method.
By treating the document processing as an important part of the RAG architecture, the developer could create AI applications that retrieve the relevant evidence and provide the answer which could be verified.
FAQs
Can RAG process scanned PDFs?
Yes. Scanned PDFs can be processed with the help of OCR to get the machine-readable text. It is recommended to check the content of the extracted text before the indexing, especially if the document contains important numbers or identifiers.
What are the best RAG chunking strategies for PDFs?
Structure-aware chunking is helpful in case of well-structured PDFs with sections and headings. The parent-child chunking helps to preserve the broader context and to get precise retrieval results. The chunking strategy depends on the document structure and evaluation results.
How can RAG improve table retrieval?
It is important to preserve the headers, row labels, values, and reporting periods of tables. In case of complicated tables, it is possible to use structured representation to improve retrieval accuracy.
What is multimodal RAG?
Multimodal RAG retrieves and processes the information from various content types (text, images). Multimodal RAG will help to answer queries related to charts and diagrams.
Is LangChain required to build a RAG pipeline?
No. LangChain is just one of the available frameworks for orchestrating the pipeline of document retrieval and language model.
References
- Docling — OCR and PDF document processing
- LangChain — Text splitting and chunking
- Weaviate — Hybrid search and exact matching
- Google Cloud Document AI — Document extraction
