DocMind — RAG Assistant (FastAPI + LangChain + Qdrant + Ollama)

A Retrieval-Augmented Generation service. Upload documents, and DocMind chunks and embeds them into the Qdrant vector database, then answers questions by retrieving the most relevant chunks and passing them to an LLM as grounding context. It keeps a short conversational memory per session for natural follow-up questions.

Runs fully local and free on Ollama by default (no API key), and can switch to OpenAI with a single environment variable.

A live retrieval-augmented answer from the running service (docker compose up + Ollama llama3.2), grounded in an ingested document and citing the source with its similarity score.

Interactive API docs at /docs.

• End-to-end RAG pipeline: ingestion → chunking → embeddings → vector search → grounded generation.
• Vector database usage (Qdrant): collection management, upserts, cosine similarity search.
• LangChain components: RecursiveCharacterTextSplitter, OllamaEmbeddings/ChatOllama, and OpenAIEmbeddings/ChatOpenAI.
• Conversational memory with a sliding window per session id.
• Provider abstraction (Ollama vs OpenAI) so the same code runs locally or in the cloud.
• Streaming responses (/chat/stream), file + inline-text ingestion, and auto Swagger docs.

flowchart LR
    U[Client] -->|"POST /documents"| ING[Ingest: extract + chunk]
    ING -->|embeddings| VDB[(Qdrant)]
    U -->|"POST /chat"| RET[Embed question + search]
    RET --> VDB
    VDB -->|top-k chunks| GEN[LLM + conversation memory]
    GEN -->|grounded answer + sources| U

docker compose up --build -d

# Pull the models once (first time only; a few GB):
docker compose exec ollama ollama pull llama3.2
docker compose exec ollama ollama pull nomic-embed-text

• Swagger UI: http://localhost:8000/docs
• Qdrant dashboard: http://localhost:6333/dashboard

# 1. Start Qdrant
docker run -d --name docmind-qdrant -p 6333:6333 qdrant/qdrant:latest

# 2. Start Ollama (or install natively from ollama.com) and pull models
docker run -d --name docmind-ollama -p 11434:11434 -v ollama:/root/.ollama ollama/ollama:latest
docker exec docmind-ollama ollama pull llama3.2
docker exec docmind-ollama ollama pull nomic-embed-text

# 3. Run the API
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt
cp .env.example .env
uvicorn app.main:app --reload

# Ingest the bundled sample document
curl -s -X POST http://localhost:8000/api/v1/documents \
  -F "file=@sample_docs/about_docmind.md"

# Or ingest raw text
curl -s -X POST http://localhost:8000/api/v1/documents/text \
  -H 'Content-Type: application/json' \
  -d '{"filename":"note.txt","text":"DocMind uses Qdrant for vector search."}'

# Ask a question (grounded in your documents)
curl -s -X POST http://localhost:8000/api/v1/chat \
  -H 'Content-Type: application/json' \
  -d '{"question":"What vector database does DocMind use?","session_id":"demo"}'

# Streaming answer
curl -N -X POST http://localhost:8000/api/v1/chat/stream \
  -H 'Content-Type: application/json' \
  -d '{"question":"Summarize how RAG reduces hallucinations","session_id":"demo"}'

export LLM_PROVIDER=openai
export OPENAI_API_KEY=sk-...
# optional: OPENAI_CHAT_MODEL, OPENAI_EMBED_MODEL

The collection dimension is derived automatically from the active embedding model, so switching providers just works on a fresh collection.

pip install -r requirements.txt
pytest

The included tests cover the ingestion/chunking logic and need no external services.

• AWS — Run the API on ECS Fargate/EKS; host Qdrant on EC2 or use Qdrant Cloud. Use Amazon Bedrock (or SageMaker endpoints) for managed LLMs/embeddings, and S3 as the document source of truth for re-ingestion.
• GCP — Cloud Run for the API, Vertex AI for embeddings/LLMs, documents in Cloud Storage, Qdrant on GKE or Qdrant Cloud.
• Azure — Azure Container Apps for the API, Azure OpenAI for models, Blob Storage for documents; Qdrant on AKS or Qdrant Cloud. (Azure AI Search is an alternative managed vector store.)

For scale, run ingestion as an async worker (SQS/Pub/Sub/Service Bus) and keep the vector DB as the shared state so the API layer stays stateless and horizontally scalable.