Back to ProjectsView Repository

Module 01

Real Estate Intelligence Platform

Built an end-to-end retrieval and generation system for Prime Lands by crawling JavaScript-rendered pages, converting unstructured content into structured corpora, and indexing it for high-quality retrieval. The platform combines multi-strategy chunking, vector search, and adaptive answer generation to return grounded, evidence-aware responses with source URLs.

Impact

Delivered a full intelligence workflow from raw web data to grounded answer generation, improving retrieval coverage, response trustworthiness, and latency for repeated queries.

Focus

Retrieval engineering, context engineering, semantic caching, corrective retrieval, and config-driven AI system design.

Highlights

  • Multi-strategy retrieval with semantic, sliding, and parent-child chunking.
  • RAG, CAG, and CRAG-based adaptive retrieval pipeline.
  • Semantic caching for low-latency repeated queries.
  • Config-driven architecture for flexible experimentation.

Tech Stack

PythonLangChainQdrantPlaywrightOpenAI / OpenRouter

Real Estate Intelligence Platform for Prime Lands

Domain-specific retrieval intelligence system that converts unstructured web data into grounded, source-backed answers.

1

Problem

Real estate data is scattered across dynamic web pages and lacks structure for reliable retrieval. Standard search fails to capture context, while LLMs without grounding produce inconsistent answers.

  • Design a system to extract, structure, and retrieve domain-specific knowledge.
  • Improve answer accuracy, traceability, and reliability.
2

Approach

  • Crawled JavaScript-rendered pages with a browser-based pipeline.
  • Converted content into structured Markdown and JSONL corpora.
  • Applied multiple retrieval strategies for stronger context quality.
  • Built a vector-backed knowledge base with adaptive retrieval-generation.
3

Context Engineering

Focused on controlling how context is selected, structured, and passed to the model.

  • Config-driven controls for chunk size, overlap, and retrieval depth.
  • Prompt templates with grounding rules and citation formatting.
  • LCEL context assembly: retriever -> formatter -> prompt -> model.
  • Evidence-aware answers with source URLs instead of free-form generation.
4

Retrieval & Chunking Strategies

  • Semantic chunking (heading-aware splits).
  • Fixed chunking (uniform size with overlap).
  • Sliding window chunking (high recall retrieval).
  • Parent-child chunking (fine-grained retrieval with broader context).
  • Late chunking (query-time refinement).
5

RAG

  • Built with LangChain LCEL pipelines.
  • Retrieves top-k relevant chunks from vector store.
  • Generates answers with structured context and grounding rules.
  • Returns evidence documents and source links.
6

CAG

  • Embedding-based similarity matching for repeated queries.
  • Two-tier cache: FAQ cache and dynamic history cache.
  • Configurable similarity thresholds and TTL.
  • Enables low-latency responses for semantically similar queries.
7

CRAG

  • Confidence scoring via keyword overlap, content richness, and chunk diversity.
  • Expands retrieval scope when confidence is low.
  • Regenerates answers with stronger context.
  • Self-corrects weak retrieval outcomes.
8

Architecture

  • Playwright-based crawler.
  • Markdown + JSONL corpus generation.
  • Multi-strategy chunking layer.
  • Embedding generation with OpenAI-compatible providers.
  • Vector indexing with Qdrant.
  • Retrieval layer with RAG, CAG, and CRAG.
9

Challenges

  • Handling dynamic, JavaScript-rendered content.
  • Balancing chunk size versus retrieval accuracy.
  • Ensuring grounded responses with reliable evidence.
  • Reducing latency for repeated queries.
  • Designing a flexible config-driven pipeline.
10

Results

  • Built a full retrieval system from raw web data to answer generation.
  • Indexed multi-strategy chunks for improved retrieval coverage.
  • Delivered grounded, source-backed responses.
  • Reduced response time through semantic caching.
  • Improved answer quality with confidence-based retrieval correction.