Refining RAG with Supabase Vector Search Filtering

Retrieval Augmented Generation (RAG) pipelines empower LLMs with external knowledge. You embed your documents, store them in a vector database, and retrieve relevant chunks based on a user query. This retrieved context then goes to the LLM. It's a powerful pattern. But a common issue arises: raw vector search can return irrelevant or out-of-scope information.

The Problem with Raw Vector Search in RAG

Vector similarity search works by finding documents whose embeddings are numerically close to the query embedding. This often works well. However, it only considers semantic similarity. It does not inherently understand other constraints or metadata.

Imagine you have a knowledge base with documentation for multiple product versions. A user asks a question about "authentication flow." Without filtering, the vector search might retrieve documents from version 1.0, 2.0, and 3.0. If the user explicitly specified "version 3.0," providing context from older versions is not just unhelpful; it can lead to incorrect answers or require the LLM to waste tokens sifting through irrelevant information.

Another example: you might have internal company policies and public FAQs in the same vector store. A query might semantically match both. You need a way to restrict searches to only public documents for external users, or only internal documents for employees.

Why Filter Vector Search Results

Filtering vector search results adds a critical layer of precision to your RAG pipeline. It allows you to apply structured constraints alongside semantic similarity.

Increased Relevance: You ensure the retrieved context strictly adheres to user-defined criteria or application logic. This means the LLM receives only truly pertinent information.
Reduced LLM Token Usage: By narrowing down the context, you send less irrelevant data to the LLM. This directly translates to lower API costs and potentially faster response times.
Improved Accuracy: An LLM with a focused, highly relevant context is less likely to hallucinate or misinterpret information.
Enhanced Control: You gain fine-grained control over what information is accessible based on user roles, document types, or other business rules.

Implementing Filtering with Supabase and `pg_vector`

Supabase, built on PostgreSQL, offers a robust solution for RAG with its pg_vector extension. The key advantage here is that your vector embeddings live in a standard relational database. This means you can combine the power of vector similarity search with the full expressive capability of SQL WHERE clauses.

First, ensure you have the pg_vector extension enabled in your Supabase project. You can do this via the Supabase Dashboard under Database -> Extensions.

Next, define your document table schema. It should include your content, the embedding, and any metadata you want to filter on. For example:

CREATE TABLE documents (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  content TEXT NOT NULL,
  metadata JSONB, -- Store structured metadata here
  embedding VECTOR(1536) -- Adjust dimension based on your embedding model
);

-- Create an index for faster vector similarity search
CREATE INDEX ON documents USING ivfflat (embedding vector_l2_ops) WITH (lists = 100);

The metadata column, using JSONB, is flexible. You can store type, source, version, author_id, created_at, or any other structured data you need.

When performing a search, you combine the vector similarity ordering with a standard SQL WHERE clause that filters on your metadata or other columns. Supabase's client libraries make this straightforward.

Practical Example: Filtering by Document Type

Let's say you have documents categorized by type, such as policy, faq, or blog_post. A user might ask a question, but you only want to retrieve faq documents.

First, insert some data with metadata:

import { createClient } from '@supabase/supabase-js';

const supabase = createClient(process.env.SUPABASE_URL, process.env.SUPABASE_ANON_KEY);

async function addDocument(content, type, embedding) {
  const { data, error } = await supabase.from('documents').insert({
    content: content,
    metadata: { type: type },
    embedding: embedding,
  });
  if (error) console.error('Error inserting document:', error);
  else console.log('Document inserted:', data);
}

// Example usage (replace with actual embeddings)
// const embedding1 = [0.1, 0.2, ...];
// const embedding2 = [0.9, 0.8, ...];
// addDocument('Our refund policy details...', 'policy', embedding1);
// addDocument('How to reset your password?', 'faq', embedding2);

Now, perform a vector search, but only for documents where metadata->>type is 'faq':

import { createClient } from '@supabase/supabase-js';

const supabase = createClient(process.env.SUPABASE_URL, process.env.SUPABASE_ANON_KEY);

async function searchFaqs(queryEmbedding) {
  const { data, error } = await supabase.from('documents')
    .select('id, content, metadata')
    .filter('metadata->>type', 'eq', 'faq') // This is the key filter!
    .order('embedding', { ascending: false, foreignTable: null, nullsFirst: false, distance: queryEmbedding })
    .limit(5);

  if (error) {
    console.error('Error searching documents:', error);
    return [];
  }
  return data;
}

// Example usage (replace with actual query embedding)
// const userQueryEmbedding = [0.5, 0.6, ...];
// const relevantFaqs = await searchFaqs(userQueryEmbedding);
// console.log(relevantFaqs);

In this query:

.filter('metadata->>type', 'eq', 'faq') is the crucial part. It uses standard PostgreSQL JSONB operators to filter documents before or during the vector similarity calculation. This ensures only FAQ documents are considered for the similarity search.
.order('embedding', { ..., distance: queryEmbedding }) performs the actual vector similarity ordering. The distance property tells pg_vector to calculate the distance between the embedding column and queryEmbedding.

This approach directly integrates your filtering logic into the database query, making it efficient and robust.

More Complex Filtering Scenarios

Supabase and pg_vector handle more than simple equality checks. You can filter by:

Date ranges: filter('metadata->>created_at', 'gte', '2023-01-01')
Numeric values: filter('metadata->>version', 'lt', 2.0)
Arrays: filter('metadata->tags', 'cs', ['important']) (using the contains operator for JSONB arrays)
User or tenant IDs: Essential for multi-tenant applications to ensure data isolation.

By carefully structuring your metadata and leveraging PostgreSQL's powerful query capabilities, you can build highly precise RAG systems.

Considerations for Performance

While filtering with WHERE clauses is powerful, ensure your filtering columns are indexed if you expect to filter on them frequently across large datasets. For JSONB columns, consider GIN indexes on specific paths if your filters consistently target the same keys.

Combining pg_vector's ivfflat or hnsw indexes for vector search with standard B-tree or GIN indexes for metadata filtering is how you achieve optimal performance in a production RAG pipeline.

Where to go from here

Start by identifying key metadata for your documents. Then, build out your Supabase schema and integrate these filters into your RAG retrieval logic. If you need help refining your RAG pipelines or implementing advanced filtering strategies, consider AI consulting.