Hacienda-ESP-RAG: Matryoshka Vector Compression & Hybrid Edge Hydration over 36,053 Tax Rulings

Why Standard Cloud RAG Breaks at 36,053 Legal Rulings

When a taxpayer or tax advisor asks an LLM a question about Spanish tax law (IRPF, IVA, Impuesto sobre Sociedades, Sucesiones), standard models frequently hallucinate repealed articles or confuse general statutes with binding administrative doctrine. In Spain, the Dirección General de Tributos (DGT) issues Binding Tax Rulings (Consultas Vinculantes) that legally bind the Tax Agency (AEAT).

However, building a production Retrieval-Augmented Generation (RAG) system over a full decade of official rulings (2016–2026: 36,053 documents, 1.05 GB of raw legal text) immediately hits two architectural walls:

• The Cloud Storage & RAM Wall: Splitting 1.05 GB of dense legal prose into standard 500-character chunks produces over 1.5 million vectors. Storing 1.5M float32(768) embeddings alongside raw text requires >6 GB of dedicated Vector RAM, immediately blowing past cloud free tiers (such as Supabase's 500 MB limit) into $70–$150/month managed vector database instances.
• The Legal Context Fragmentation Problem: A single DGT ruling spans 5 to 15 pages structured into Antecedentes de Hecho (facts), Normativa Aplicable (statutes), and Contestación (binding legal conclusion). Retrieving isolated 500-character snippets gives the LLM the facts without the ruling, or the ruling without the conditions.

The live educational benchmark at hacienda.mezapps.com comparing full 768D vector retrieval against Matryoshka-sliced 128D, 256D, and 512D embeddings in real time.

Scraping 10 Years of Spanish Tax Doctrine & High-Signal Legal Chunking

The official Spanish Ministry of Finance doctrine database (PETETE / Knosys) has no public REST API or bulk dataset export. It relies on stateful server-side sessions and legacy HTML forms. To build a complete, reproducible legal corpus from 2016 to 2026, I engineered a custom asynchronous scraping and ingestion pipeline (scraper_petete.py):

Contextual Header Injection

When a long ruling is split into multiple chunks, secondary chunks normally lose awareness of which ruling number, tax date, or governing body they belong to. Before computing embeddings on the GPU, the chunker dynamically injects a rich metadata header into every single fragment:

# Dynamic contextual prefix injected into every 4,500-char legal chunk before GPU embedding
header_prefix = (
    f"[Consulta Vinculante DGT {num_consulta} | "
    f"Fecha Salida: {fecha_salida} | "
    f"Órgano: {organo} | "
    f"Normativa: {normativa[:180]}]\n"
)
enriched_chunk_text = header_prefix + clean_legal_body

This single data-engineering decision reduced total vector rows from an estimated ~500,000 down to ~150,000 high-density chunks while dramatically improving retrieval precision for specific statutory articles (e.g., Artículo 38.1 Ley 35/2006 LIRPF).

The "Russian Doll" Effect: Cutting Vector Memory by 83.3% Without Re-Indexing

In traditional embedding models (such as text-embedding-ada-002 or standard BERT derivatives), semantic information is distributed uniformly across all 768 or 1,536 vector dimensions. Dropping the last 80% of coordinates destroys the vector's meaning.

Matryoshka Representation Learning (MRL)—used to train Google's EmbeddingGemma—optimizes a nested multi-scale loss function across prefixes of the vector (128D ⊂ 256D ⊂ 512D ⊂ 768D) simultaneously during training. Just like nested Russian dolls, the neural network is forced to pack the highest-variance semantic concepts (the core tax type, transaction structure, and statutory article) into the first 128 dimensions, while dimensions 129..768 encode progressively finer syntactic nuances.

The Mathematical Catch: Why L2 Re-Normalization is Mandatory

A full 768-dimensional embedding generated by EmbeddingGemma lies on the surface of a 768D unit hypersphere (‖v1:768‖2 = 1). When we slice the vector to its first d ∈ {128, 256, 512} coordinates, the truncated vector v1:d falls inside the d-dimensional unit sphere (‖v1:d‖2 < 1). To preserve exact cosine similarity ordering via inner-product / cosine operators in pgvector, both the query vector and the database sub-vector must be L2 re-normalized after slicing:

def slice_and_normalize_mrl(vec_768: list[float], dim: int) -> list[float]:
    """
    Matryoshka (MRL) prefix truncation + L2 hypersphere re-normalization.
    Ensures exact cosine geometry in R^dim after slicing from R^768.
    """
    sliced = vec_768[:dim]
    norm = math.sqrt(sum(x * x for x in sliced))
    if norm > 0:
        return [x / norm for x in sliced]
    return sliced

On the cloud side, rather than duplicating tables for 128D, 256D, 512D, and 768D, we store a single halfvec(768) (16-bit float) column in Supabase PostgreSQL and perform dynamic SQL sub-vector slicing inside our custom RPC function:

-- Dynamic Matryoshka slicing inside PostgreSQL pgvector (0 duplicate storage)
CASE
    WHEN match_dimensions = 128 THEN
        1 - (subvector(c.embedding, 1, 128)::halfvec(128) <=> query_embedding::halfvec(128))
    WHEN match_dimensions = 256 THEN
        1 - (subvector(c.embedding, 1, 256)::halfvec(256) <=> query_embedding::halfvec(256))
    WHEN match_dimensions = 512 THEN
        1 - (subvector(c.embedding, 1, 512)::halfvec(512) <=> query_embedding::halfvec(512))
    ELSE
        1 - (c.embedding <=> query_embedding::halfvec(768))
END AS similarity

Zero Text in the Cloud: Pairing Supabase pgvector with a Local NVIDIA Jetson Orin Nano

Even after compressing embeddings with halfvec and Matryoshka slicing, uploading 1.05 GB of raw legal text into a managed cloud Postgres database would immediately exceed the 500 MB Free Tier limit. To keep cloud recurring costs at $0.00/month while serving full 15-page rulings, I designed a Hybrid Edge + Cloud "Local Hydration" Architecture:

Exposing a Physical Home-Lab GPU Server Safely to the Public Internet

Because the backend runs 24/7 on a physical NVIDIA Jetson Orin Nano inside a residential network and invokes paid LLM synthesis APIs, exposing it publicly required defense-in-depth engineering across the network, application, and OS layers:

Designed for Both Engineers & Non-Technical Visitors (ES / EN)

Most RAG benchmarks are static Jupyter notebooks. I built the frontend (hacienda.mezapps.com) in Next.js 16 (App Router) + Tailwind CSS as an interactive, self-explanatory laboratory:

• Interactive Matryoshka Slicing Bar: Visitors can switch dynamically between 128D (17% vector size), 256D (33%), and 512D (67%) and watch the active memory bar, byte footprint per chunk, and Recall@10 overlap update live against the 768D (100%) baseline.
• Cross-Lingual Legal Synthesis (ES / EN Toggle): International recruiters and engineers who don't speak Spanish can switch the interface to English (EN). Curated real-world tax scenarios (from startup corporate tax rates to Beckham Law impatriate rules and primary residence reinvestment) are translated into natural English, and the backend instructs Gemini to write the legal explanation in English while preserving exact Spanish statutory references.
• Anti-Hallucination Citation Modal (📄 Vxxxx-xx): Every binding ruling cited in the AI's response is parsed via regex into an interactive badge. Clicking any 📄 V0714-26 pill opens a modal displaying the complete official text of the ruling hydrated directly from the Jetson's SQLite database, with the exact chunk retrieved by the vector engine highlighted in context.
• Mobile-First 3-Way Tab Switcher: On smartphones, comparing two wide RAG columns side-by-side is unreadable. A sticky mobile tab bar (🪆 128D (-83%) | 🔵 768D (100%) | ⚖️ Both) and a swipe-friendly bottom-sheet modal make the full benchmark effortless to explore on mobile devices.