Introduction

We’ve all seen the standard RAG demo.

Upload a few PDFs. Ask a question in natural language. Get back a polished answer in a chatbot window.

It looks impressive for about five minutes.

Then someone asks a real business question:

• “What’s the vehicle number for e-way bill FM-GST-2026-2001?”

• “Show me the part number for Seat Foam Cushion.”

• “What is the total amount for invoice TF/2026-27/001?”

• “Where is the registered office of RIL?”

And suddenly the problem is not “generate a nice answer.”

The problem is retrieve the right document, the right page and the right evidence with high precision.

That is the gap this project was built to solve. This is not a generic chatbot for PDFs.

It is a retrieval-first document intelligence system for industrial and operational workflows.

The goal is simple:

Given a business-style query over invoices, bills of materials, e-way bills, and financial PDFs, return the correct document and evidence quickly, reliably, and in a format a real user can trust.

Follow Here:

Github: Click Here
X/Twitter: Click Here
LinkedIn: Click Here

The Actual Problem

In a factory or operations setting, most document questions are not open-ended. They are structured, messy, and annoyingly specific.

A finance person does not want “a summary of the invoice.”
They want:

• the invoice matching a number

• the line item matching a description

• the page containing the evidence

• the exact value, code, or supplier

Likewise, a procurement or production user searching a BOM is not asking for broad semantic understanding.

They are asking:

• “Which material code matches this component?”

• “Which BOM contains this assembly?”

• “What revision is this part on?”

Traditional RAG discussions usually start with generation. This problem starts with retrieval quality.
If retrieval is wrong, generation just makes the wrong answer sound more confident.
So I deliberately framed this as:

evidence-backed retrieval first, optional generation second.
That one design choice shaped almost everything else in the project.

Why I Didn’t Build “Traditional RAG”

The typical RAG mental model looks something like this:

1. Chunk documents

2. Embed them

3. Put them in a vector database

4. Retrieve top-k chunks

5. Ask an LLM to answer

That pattern is useful, but it breaks down quickly on operational documents.
Why?
Because factory and finance PDFs contain:

• identifiers like invoice numbers and e-way bill numbers

• amounts, tax values, and totals

• table rows and line items

• material codes and part numbers

• short layout-bound evidence instead of long narrative passages

Those are not purely semantic retrieval problems.

In fact, many of them are lexical, structured, or exact-match problems disguised as natural-language queries.
That is why this system does not rely on embeddings alone.
Instead, it combines:

• exact identifier routing

• lexical chunk search

• keyword retrieval

• vector retrieval for fuzzy queries

• reranking on top of merged evidence

The LLM is optional and comes after retrieval.

That means the system behaves more like a document search engine with AI support than a chat interface pretending to be one.

What I Built

At a high level, the system supports:

• PDF ingestion from files, folders, or a shared inbox

• text extraction and lightweight metadata enrichment

• evidence chunking across text blocks, lines, table rows, and header fields

• multi-store retrieval using PostgreSQL, OpenSearch, and Qdrant

• query routing into exact, lexical, mixed, and hybrid paths

• evidence-first responses through FastAPI and Streamlit

• optional Gemini-based summarization on top of retrieved evidence

• evaluation for both retrieval and extraction quality

The stack includes:

• FastAPI for the backend API

• Streamlit for the business-facing UI

• PostgreSQL for metadata, chunk storage, and query logs

• OpenSearch for keyword retrieval

• Qdrant for vector retrieval

• Redis for query-result caching

• SentenceTransformers as the default embedding backend

• Gemini as an optional post-retrieval answer layer

• Docker Compose for local deployment

The Architecture

The system has three main flows:

1. Ingestion

When a PDF enters the system

:

1. The file is hashed for checksum-based duplicate detection

2. Text is extracted with PyMuPDF

3. The document is classified into types like invoice, BOM, or e-way bill

4. Lightweight metadata is extracted

5. The original PDF is copied into local storage

6. Multiple evidence units are built:

   • text chunks

   • line chunks

   • table-row chunks

   • header-field chunks

7. Chunks are written to Postgres

8. Keyword representations are indexed into OpenSearch

9. Vector embeddings are indexed into Qdrant

The key idea here is that not all evidence in business PDFs should be treated as generic text chunks.

A line item and a header field behave very differently in search.

2. Query Routing

When a user submits a query, the router decides whether the search should behave like:

• exact_match
  for strong identifiers like invoice numbers or document IDs

• lexical
  for row-level, numeric-heavy, or highly specific queries

• mixed
  for queries that benefit from both lexical and semantic signals

• hybrid
  for fuzzier text queries

This is one of the most important design choices in the system.

I did not want every query to be treated as “vector search + prompt.”
FM-GST-2026-2001 should not go through the same retrieval path as “registered office for RIL”.

3. Retrieval and Answering

Depending on the route, the system combines signals from:

• PostgreSQL chunk search

• OpenSearch keyword search

• Qdrant vector search

Then it performs weighted score fusion and reranking.
The answer layer does not blindly generate text.

It follows a priority cascade:

1. deterministic exact-match answer when possible

2. extractive row answer for line-item style queries

3. optional Gemini summary if enabled

4. extractive fallback using top-hit evidence

That keeps the system grounded in evidence instead of turning retrieval misses into fluent hallucinations.

A More Practical Ingestion Workflow

One part I cared about beyond the search itself was how non-technical users would actually use the system.

In a real office, users are not going to open an IDE and run CLI commands to ingest PDFs.

So I added an inbox-style workflow:

• users drop PDFs into a shared incoming folder

• Streamlit shows the configured inbox path

• the UI triggers bulk ingestion through the API

• duplicates are skipped automatically using checksum-based deduplication

This is much closer to how the system would be used in a small factory or finance team.

Long term, I would evolve this into:

• incoming/

• processed/

• failed/

plus a scheduled or background ingestion worker.
That is a better operational pattern than telling users to manually manage file paths.

What the Benchmark Says

I evaluated the system on a held-out benchmark covering invoices, BOMs, e-way bills, and financial PDFs.

Retrieval Results

• Recall@1: 86.1%

• Recall@3: 97.2%

• Recall@5: 100%

• MRR: 0.922

Those are strong enough for a practical retrieval-first v1.
They suggest the system is doing what it was designed to do:

find the correct document and supporting evidence reliably.

Extraction Results

• Documents passed: 0/18

• Field accuracy: 0.5591

• Line item accuracy: 0.7222

  This matters.

Because it forced an important correction in how I describe the project.
This is not a universal, format-independent document extraction engine.

It is strongest as:

• an evidence-backed retrieval system

• a production-style document search application

• a realistic RAG architecture for operational PDFs

That distinction makes the project more honest and, in my opinion, more useful.

What Worked Better Than Expected

A few things ended up mattering more than I initially expected:

1. Exact-match routing

Business document search is full of identifiers.

The moment I generalized document-ID detection and routed those queries into exact-match behavior, retrieval quality improved sharply on those cases.

2. Row-level evidence

Generic chunking is not enough for invoices and BOMs.
Adding table-row and header-field evidence made the system much better at line-item style retrieval.

3. Retrieval fusion

Lexical and semantic retrieval are not enemies.
They solve different parts of the problem.

The best behavior came from routing queries appropriately and then fusing signals rather than betting everything on one retrieval mode.

What Was Harder Than Expected

Three things were more painful than the happy-path architecture diagram suggests.

1. Layout-sensitive extraction

Metadata extraction on operational PDFs is surprisingly fragile.

Even when the system retrieves the right document, robustly extracting supplier, buyer, amount, and line items across unseen layouts is still hard.

That is why the retrieval story is much stronger than the extraction story right now.

2. Docker and dependency behavior

The app side was easier than the packaging side.

I hit repeated issues with Docker rebuild latency and dependency resolution. Fixing layer invalidation, caching behavior, and CPU-only Torch resolution took real effort.

It was a useful reminder that “production-style” work is often more about the connective tissue than the main algorithm.

3. Synchronous ingestion

The inbox ingestion flow works, but synchronous ingestion through a UI is not the end state.

Cold starts, model load time, and indexing latency mean that background ingestion is the right next step.

Why This Project Matters to Me

I did not want to build another generic “chat with your PDF” demo.

I wanted to build something that reflects how messy business AI systems actually are:

• real documents

• mixed retrieval modes

• evidence and citations

• deployment trade-offs

• evaluation that exposes weaknesses instead of hiding them

This project ended up teaching me something I think applies well beyond RAG:

The hardest part is usually not the model. It is figuring out what the product should actually do, what part of it must be trustworthy, and what trade-offs you are willing to make.

In this case, the right answer was:

-build a strong retrieval system first
-measure it honestly
-and let generation stay downstream of evidence

What I’d Improve Next

If I keep pushing this forward, the next improvements are clear:

• asynchronous scheduled ingestion

• incoming / processed / failed folder lifecycle

• richer ingestion manifests and operational reporting

• stronger OCR support for scanned documents

• better financial statement retrieval

• confidence-aware extraction fallback instead of purely deterministic extraction

• stronger integration testing across all stores and APIs

In other words: less demo polish, more operational depth.

Final Takeaway

If you are building document AI for a real workflow, start by asking:

What is the actual task?

If the task is:

• retrieve the right document

• show the right page

• ground the answer in evidence

• handle identifiers, amounts, and line items correctly

then a traditional “vector search + prompt” RAG setup is often the wrong mental model.

This project worked better once I stopped treating it as a chatbot problem and started treating it as a document retrieval system with AI-enhanced evidence handling.

That framing changed the architecture, the evaluation, and the final product.
And I think it made the system much better.

Here’s the github repo, Note: there are 2 branches, namely master and cloud_deploy.

Moving from Docker Compose to AWS EKS was a massive shift in mindset. You quickly realize that in the cloud, everything is a moving part. Nodes disappear, IP addresses change and permissions are the difference between a working system and a wall of 403 errors.

AWS Design and config doc

Complete System Design

1. The Cloud Stack

The final cloud version used:

• Amazon EKS for Kubernetes runtime

• Terraform for infrastructure provisioning

• Amazon ECR for Docker images

• GitHub Actions for CI/CD

• Amazon Bedrock for chat and embeddings

• Redis inside the cluster for cache and task state

• Chroma inside the cluster for semantic recall

• Amazon S3 for durable artifacts

• EBS-backed PVCs for persistence

One important design choice: I did not go for every managed service AWS offers. I kept the stack lean so it could be built, explained and torn down without becoming a never ending cloud bill.

When you run things locally with Docker Compose, life is simple. You share a folder between containers, and they talk to each other like they are in the same room. But Kubernetes is more like a crowded city. Pods are ephemeral; they are born and die constantly. If your /train-child task saves a model to a local folder and that pod gets rescheduled, your model is gone forever.

To bridge this gap, we had to re-engineer how the system holds onto its state:

• The Entryway (AWS ALB): We used the AWS Load Balancer Controller to set up an Application Load Balancer. This gives us a single, stable DNS name. It handles incoming traffic and routes it to our FastAPI services, no matter which node they are currently sitting on.

• Storage that Survives (Amazon S3): We swapped local output folders for S3 buckets. Now, every time a StockLSTM finishes training or a monitor report is generated, it gets pushed to S3. This means our inference engine can pull the latest weights from anywhere in the cluster without worrying about disk persistence.

• Redis runs as a Kubernetes Deployment plus Service in k8s/redis.yaml. Chroma runs as a Kubernetes Deployment plus Service in k8s/chroma.yaml. Their persistent data is backed by PVCs, which in the final cloud setup use EBS, not an AWS managed cache service.

2. The Compute Strategy: Managing the Cloud Tax

One of the biggest traps in cloud ML is compute cost. You do not want to burn money running a lightweight Streamlit dashboard on a giant GPU node just because it sounds impressive.

For this project, we kept the AWS footprint intentionally lean:

• one EKS cluster

• one managed node group

• a small number of general purpose EC2 instances

• only the services we actually needed

• no unnecessary managed extras that would complicate the stack or inflate the bill

That simplicity mattered. It let us focus on whether the application actually worked in a cloud environment instead of getting buried in infrastructure sprawl.

The key idea was this: every part of the system should justify its existence.

If a service was only useful locally, it had to be replaced or removed.
If a folder only existed because Docker Compose made it easy, we had to think harder about persistence.
If a component could run in-cluster without a managed AWS service, we preferred that path for the first version.

That is why Redis and Chroma stayed in Kubernetes, and why we used EBS-backed PVCs instead of jumping straight to more expensive managed services.

3. Rebuilding The Runtime For Kubernetes

Docker Compose gives you a comforting illusion. Everything is on one machine, the network is flat, and storage feels permanent enough.

Kubernetes strips that comfort away.

Pods are ephemeral. Nodes can be replaced. IPs change. Service names matter. Storage has to be treated as part of the architecture, not an afterthought.

That meant the runtime had to be rethought in a few important ways.

The Front Door

For the UI, we used a Kubernetes LoadBalancer service for the frontend. That gave us a stable public entry point in AWS without overengineering the exposure layer.

The frontend became the public face of the system. From there, it talked to FastAPI inside the cluster.

FastAPI As The Control Plane

FastAPI became the central orchestration layer for:

• training

• prediction

• analysis

• monitoring

That was a good fit because the backend was already the brain of the project. It knew how to train child models, produce forecasts, pull news, evaluate sentiment, and run the custom monitoring pipeline.

In the cloud version, FastAPI did not just serve endpoints. It became the coordinator for the whole workflow.

Redis For State

Redis stayed in-cluster.

This made sense for the first AWS version because the application only needed a practical cache and task state store. We were not trying to solve distributed systems theory on day one. We just needed the system to keep working reliably enough for a demo and a real test.

Chroma For Semantic Recall

Chroma also stayed in-cluster, with persistent storage backed by PVCs.

That decision kept the architecture simple while still preserving the semantic cache behavior that the app relied on.

The important part was persistence. If the pod restarted, we still wanted the semantic store to survive. That is exactly why we paired Chroma with storage backed by EBS.

S3 For Durable Artifacts

This was one of the most important cloud decisions.

Local folders are fine until the pod disappears. In Kubernetes, local filesystem state is fragile. So for anything that mattered beyond the lifetime of a single container, we moved to S3.

That included:

• trained model artifacts

• scalers

• analysis outputs

• monitoring summaries

• other durable files that needed to survive restarts

This gave the system a proper cloud-native memory for important outputs.

4. Bedrock Replaced Local Model Serving

The other big change was the AI layer.

Instead of relying on a local model server, the cloud version used Amazon Bedrock for:

• chat model calls

• embeddings

That solved a lot of operational problems at once.

There was no need to run a separate local inference stack.
There was no need to keep a model server alive in the cluster.
And there was no need to depend on one more container that could fail independently.

The FastAPI app just needed AWS credentials and the right Bedrock model IDs. Once that was in place, the rest of the system could call the model layer through a normal AWS path.

That made the architecture much cleaner for the cloud version.

5. Terraform Made The Whole Thing Reproducible

Once the runtime shape was clear, Terraform took over the infrastructure side.

Checkout the terraform directory

This was important because cloud deployment gets messy fast if you create things by hand and then try to remember how you did it later.

We used Terraform to provision:

• the VPC

• public and private subnets

• the EKS cluster

• the managed node group

• ECR repositories

• the S3 artifact bucket

• the IAM pieces needed for EKS and GitHub Actions

• the EBS CSI driver path for persistent volumes

That gave us something much better than a one-off cloud setup.

It gave us a repeatable deployment path.

If I wanted to recreate the stack, I did not need a memory dump from my browser tabs. I could read the Terraform and know what would happen.

That is a huge difference when you are trying to turn a prototype into something real.

6. GitHub Actions Became The Deployment Conductor

Once the infrastructure existed, GitHub Actions became the thing that tied everything together.

We ended up with three workflows:

• ci.yml

• build-and-push.yml

• deploy-eks.yml

  

The ci.yml workflow handled sanity checks on the cloud_deploy branch.
The build-and-push.yml workflow built the Docker images and pushed them to ECR.
The deploy-eks.yml workflow applied the Kubernetes manifests to the cluster.

That sounds simple, but the actual path had a few rough edges.

For example:

• the Dockerfile path had to point to the right subdirectory

• the feature_store/ folder had to exist in Git for the image build to work

• the deploy workflow had to generate secrets carefully

• the EKS access entry had to be created so the GitHub role could actually deploy

In other words, the CI/CD pipeline was not just “automation.” It was the glue that made the whole cloud setup operational.

7. Where Monitoring Fit In

This project has two kinds of monitoring, and it is worth separating them.

App-level monitoring

This is the custom ticker monitoring logic in the backend.

It checks things like:

• system health

• regime drift

• response quality

• analysis quality

This is the real product logic behind the /monitor/{ticker} endpoint.

Infrastructure monitoring

This is the Prometheus and Grafana side.

Prometheus scrapes backend metrics.
Grafana can visualize them.

That part is for observability, not the actual financial monitoring logic.

So when I say the system has monitoring, I do not mean just dashboards. The app itself has a built-in monitoring layer that reasons about ticker behavior, regime changes and report quality.

That distinction matters because it reflects the difference between “the service is up” and “the model output is trustworthy.”

8. The Debugging Was Part Of The Build

A cloud migration is never just a deployment exercise. It is also a long conversation with everything you accidentally assumed.

Some of the issues we hit were small, but each one taught something real:

• the FastAPI image initially missed the monitoring/ package

• Chroma crashed because service environment variables leaked into its config

• the first build workflow pointed at the wrong Dockerfile path

• the second build failed because feature_store/ had not been committed

• the deploy workflow failed because the generated secrets file path did not exist yet

• FastAPI could not call Bedrock until AWS credentials were available inside the pod

Each issue was a reminder that cloud systems fail at the seams.

And those seams are where the important engineering happens.

9. Important Learnings

• Build and validate the ML system locally first, then move it to AWS.

• Keep the first cloud setup small. Add nodes and services only after the base path works.

• Set a billing alert early, even for PoC work.

• Treat monitoring as part of the product, not an afterthought.

• Use caching to reduce inference latency and cloud cost.

• Terraform is worth it because it makes the infrastructure repeatable.

• MLOps is not just the model. It includes backend, infra, deployment, and observability.

• Tear down AWS resources when you are done to avoid surprise charges.

• Do not chase a perfect system on day one. Ship a working version, then improve it.

Visit my github

Visit my twitter

Visit my linkedin

But if you want to build something that actually works in a production-like environment, you need more than a model. You need an production grade ML System.

Over the last few weeks, I’ve been building this system. It’s an end-to-end system that handles everything from parent-child transfer learning to agentic news analysis. Here is how it’s put together.

This phase mainly focuses on system design and how to put everything together. Next phase will likely involve K8/cloud deployment.

Note:
This project is an exercise in ML Systems Engineering, not a quest for the perfect financial signal.

While I’ve focused on model quality and performance, the primary goal here is to demonstrate a POC for an end to end pipeline. The emphasis is on the engineering decisions, system design the “connective tissue” that makes an ML model functional in a production like environment, rather than simply maximizing prediction accuracy.

Github Repo

Introduction:

If you have ever tried to deploy a machine learning model, you know the “Notebook to Production” gap is more like a canyon. For this project, I set out to bridge that by building a pipeline that manages itself.

The idea is simple: a user asks for a forecast on a ticker, and the system coordinates a small army of services to give an answer that is both mathematically sound and narratively backed.

It’s not trying to beat hedge funds (good luck with that). It’s for learning, experimentation, personal trading ideas or as a starting point for something bigger.

The Stack: More Than Just Python

The core of this system is a distributed architecture designed to handle data ingestion, training and real-time inference.

• The Brain (FastAPI): Everything is an asynchronous task. You hit /train-child, get a task ID, and let the backend do the heavy lifting in the background.

• The Memory (Redis & Chroma): Compute is expensive. If someone asks for the same analysis twice, Redis pulls the exact cache. If they ask for something similar, Chroma provides a semantic cache of previous LLM insights.

• The Context (Ollama & LangGraph): A price chart is blind to news. I built an agent layer that fetches headlines from NewsAPI and Finnhub, then uses an LLM to “read” the sentiment and weigh it against the numerical forecast.

Why the “Parent-Child” Strategy?

Most developers train one model per stock. That is a maintenance nightmare. Instead, I implemented a hierarchical training flow.

I start with a Parent Model. A robust model trained on broad market sector data to capture general “market physics.” When I want to predict a specific ticker like INFY.NS or AAPL, I fine-tune a Child Model on that specific asset. It is faster, more data-efficient and inherits a baseline understanding of how markets move.

But why use an LSTM for this? In a world obsessed with Transformers (I’m too), LSTMs remain a pragmatic choice for time series forecasting for a few reasons:

1. Memory over Attention: Stocks are sequential. LSTMs are explicitly designed to maintain a hidden state that carries information across time steps, making them naturally suited for the lead lag effects in price action.

2. Data Efficiency: Transformers are notoriously data hungry. Since we are often dealing with limited high quality daily bars for specific tickers, an LSTM converges much faster and is less prone to overfitting on small datasets.

3. The Vanishing Gradient Solution: Compared to older RNNs, the LSTM’s gated architecture (Input, Forget and Output gates) allows it to decide which historical events, like a massive earnings beat three months ago, are still relevant to today’s price.

Keeping an Eye on the Regime

The rules of the market change. A model built for a bull market is a liability during a crash. I didn’t want to just hope the model was still accurate, so I built a Monitoring Runner.

Every time you hit /monitor/{ticker}, the system runs a check for Regime Drift. It compares current market volatility and returns to what the model saw during training. If the market regime has shifted significantly, the system flags it in a JSON artifact. Simply put, it monitors its own relevance.

Observability

If you aren't measuring it, you aren't managing it. The whole pipeline exports metrics to Prometheus, which I visualize in Grafana.

Custom monitoring that checks:

• Overall system health

• Market regime drift (is the current market behaving differently from training?)

• Quality of the generated analysis

All of this runs together with Docker Compose, so spinning up FastAPI, Redis, Prometheus, Grafana, and the Streamlit dashboard is just one command.

Frontend

There’s a simple Streamlit app that lets you:

• Trigger training, prediction, analysis, or monitoring

• View the latest forecast chart

• Read the generated analysis

• Check monitoring results in cards and tables

What can be done better?

• Train a larger LSTM model.

• Use Evidently AI for more professional and accurate drift detection.

• Deployment to cloud (Next phase, stay tuned!)

Major Learnings:

• Every Decision Starts with “It Depends”: There is no perfect architecture, only trade-offs. We chose an LSTM over a Transformer and Redis over a standard DB because those tools fit the specific latency and data constraints of this project.

• Solve the Business Problem, Not the Hype: At the end of the day, what matters is the problem you are solving. For this pipeline, the problem wasn’t just predicting a price, it was providing actionable, monitored financial insights.

• Stack Alignment: Your current stack must support your ultimate goal. By using FastAPI and Docker, it is a portable, scalable service that actually works in a production like environment.

• The Invisible Infrastructure: Trust me, training the ML model will be the easiest part of your job. The real value and often the real difficulty lies in the "connective tissue." The data flow, caching strategy and the observability are what keep the system from flying blind in production.

What’s Next?

We are not done yet. it’s a foundation. The repo is designed so you can swap out the LSTM for a Transformer or change the LLM provider without breaking the plumbing. Next phase will involve deploying this whole thing to cloud, so stay tuned!!

If you found this useful, follow me on GitHub, X and linkedin. You can find the full repo and setup instructions at the link below. Go ahead and fork it, clone it, build it, break it and build it again yourself. Because the truth is, until you’ve built the thing yourself, you don't really understand it.

Visit my github

Visit my twitter

Visit my linkedin