PageIndex

Vectorless, Reasoning-based RAG

📑 Explore PageIndex on GitHub

Check out the official repository for Vectorless, Reasoning-based RAG by VectifyAI.

1. Executive Summary: What is PageIndex?

PageIndex is a reasoning-based RAG framework that fundamentally shifts away from traditional semantic search by eliminating Vector DBs and artificial chunking. Instead, it relies on:

❌ No Vector DB (Database)
❌ No Artificial Chunking

✅ Reasoning-based Retrieval
✅ Human-like Hierarchical Navigation

It builds a hierarchical Tree Index directly from long, complex documents and utilizes LLMs to perform reasoning-driven searches, effectively simulating how human domain experts navigate extensive texts.

2. Problem Statement

Limitations of Traditional Vector RAG

Retrieval is based purely on semantic similarity.
Similarity ≠ Relevance: High similarity scores don't guarantee the correct context.
Artificial chunking breaks the natural context and document flow.
Lack of traceability and explainability (the "black box" effect).
"Vibe retrieval": Opaque and approximated matching.

The Professional Document Challenge

PageIndex is ideal for highly structured, long-form professional documents:

Financial Reports (SEC filings, Earnings)
Regulatory Disclosures
Technical Manuals
Legal Documents
Extensive Textbooks
Any PDF exceeding a single LLM's context window.

3. Solution Architecture

3.1 Core Concept: Tree Index

PageIndex transforms a monolithic, extensive PDF into a structured JSON tree format. Each node explicitly represents a logical section of the document.

{
  "title": "Financial Stability",
  "node_id": "0006",
  "start_index": 21,
  "end_index": 22,
  "summary": "...",
  "nodes": [...]
}

Each node explicitly contains the section's Title, the exact Page Range, a detailed Summary, and its nested Sub-nodes.

3.2 Execution Pipeline

Step 1: Hierarchical Index Generation

Constructs a "Table of Contents" style tree. The segmentation occurs naturally by following the document's inherent logical structure rather than arbitrary character limits.

Step 2: Reasoning-Based Retrieval

Instead of mathematical similarity, the LLM actively performs a "tree search." It iteratively evaluates relevance and strategically selects context guided by the specific query objective.

4. Technical Differentiators

No Vector DB

Completely bypasses embeddings entirely and drops traditional cosine similarity search architectures.

No Chunking

Maintains the natural semantic sequence and fully respects the overarching document hierarchy.

Human-like Retrieval

Simulates an expert's workflow: scans the table of contents, explores relevant sections sequentially, and refines the search progressively.

Explainability

Features explicit references to pages and structural sections. The entire thought process and retrieval mechanism is fully auditable.

5. Deployment Options

Open Source (Self-host)

Run locally down to the metal with minimal dependencies:

pip3 install --upgrade -r requirements.txt

CHATGPT_API_KEY=your_openai_key_here

python3 run_pageindex.py --pdf_path /path/to/document.pdf

Cloud Service

ChatGPT-style managed platform
Native API Integration
Integration via MCP (Model Context Protocol)

Enterprise Deployment

Private Deployments
On-premise execution
White-label custom enterprise configurations

Comparison: Traditional RAG vs PageIndex

Feature	Traditional Vector RAG	PageIndex
Vector DB Requirement	Requires Vector DB (✔)	No Vector Database required (❌)
Artificial Chunking	Relies on rigid size chunking (✔)	Maintains structural integrity (❌)
Retrieval Paradigm	Mathematical Similarity-based	Reasoning-based
Explainability & Audit	Low / Opaque	High / Fully Auditable
Hierarchical Navigation	Lacks structure awareness (❌)	Tree-based traversal (✔)

Performance Benchmark highlight

On the rigorous FinanceBench, the Mafin 2.5 system (powered directly by PageIndex) achieved a staggering 98.7% accuracy, significantly outperforming traditional vector-based pipelines on complex financial reasoning tasks.

Ideal Use Cases

Financial Analysis (SEC Filings, Earnings) Technical Document QA Audit & Compliance Deep Knowledge Extraction

Target Audience

AI Architects GenAI Engineers Financial Analytics Teams LegalTech Professionals Enterprise Knowledge Systems Managers

Test Your Knowledge

Score 8/10 or higher to pass