Chapter 10 · AI Architect

Vector-DB design and metadata filtering

vector store · metadata filter · top-k similarity retrievalARCH-90268 of 84

A toy vector store just ranks everything by similarity and returns the closest. A production vector store does something crucial first: it filters by metadata, then ranks the survivors.

Why filter first? Because "nearest" isn't enough. You only want this tenant's documents, only the ai namespace, only docs from this year. Searching across everything and ranking by similarity alone returns confident, well-matched, *wrong-context* results. The order is the whole lesson:

1. Filter on metadata — drop anything that fails the where clause ({"ns": "ai", "year": 2026}).
2. Score only the survivors with your similarity function.
3. Top-k — sort by score descending and take the best k.

The metadata match is an AND across every key: every key in where must match, and an empty where matches everything. For determinism, break score ties by id (so the same query always returns the same order).

Here similarity is a toy overlap score and the store is an in-memory list. In production this is pgvector or Pinecone, where the metadata filter becomes a WHERE clause pushed down *before* the approximate-nearest-neighbour search — same order, real engine.

Syntax

def passes(meta, where):
    # AND across every key; empty where matches all
    return all(meta.get(k) == v for k, v in where.items())

def query(vec, k, where):
    candidates = [d for d in DOCS if passes(d["meta"], where)]   # 1. filter
    scored = [(similarity(vec, d["vec"]), d["id"]) for d in candidates]  # 2. score
    scored.sort(key=lambda p: (-p[0], p[1]))                     # score DESC, id ASC
    return [doc_id for _score, doc_id in scored[:k]]             # 3. top-k

Worked examples

The metadata filter is an AND; empty where matches all

def passes(meta, where):
    for key, val in where.items():
        if meta.get(key) != val:
            return False
    return True

print(passes({"ns": "ai", "year": 2026}, {"ns": "ai"}))            # one key matches
print(passes({"ns": "ai", "year": 2025}, {"ns": "ai", "year": 2026}))  # year differs
print(passes({"ns": "ai"}, {}))                                   # empty where
# Output:
# True
# False
# True

Filter on metadata FIRST, then rank by similarity

def similarity(a, b):
    return len(set(a) & set(b))      # toy overlap score

DOCS = [
    {"id": "d1", "vec": ["ai", "agent", "loop"],  "meta": {"ns": "ai", "year": 2026}},
    {"id": "d2", "vec": ["ai", "rag"],            "meta": {"ns": "ai", "year": 2025}},
    {"id": "d3", "vec": ["soil", "plants"],       "meta": {"ns": "home", "year": 2026}},
    {"id": "d4", "vec": ["ai", "agent", "rag"],   "meta": {"ns": "ai", "year": 2026}},
]

def passes(meta, where):
    return all(meta.get(k) == v for k, v in where.items())

def query(vec, k, where):
    cands = [d for d in DOCS if passes(d["meta"], where)]
    scored = [(similarity(vec, d["vec"]), d["id"]) for d in cands]
    scored.sort(key=lambda p: (-p[0], p[1]))
    return [doc_id for _s, doc_id in scored[:k]]

# only ns=ai, year=2026 survive the filter (d2 is 2025, d3 is home)
print(query(["ai", "agent", "rag"], 2, {"ns": "ai", "year": 2026}))
# Output:
# ['d4', 'd1']

Tie-break by id keeps results deterministic

def similarity(a, b):
    return len(set(a) & set(b))

docs = [
    {"id": "b", "vec": ["ai"]},
    {"id": "a", "vec": ["ai"]},
]
# both score 1 — without the id tie-break the order would be arbitrary
scored = [(similarity(["ai"], d["vec"]), d["id"]) for d in docs]
scored.sort(key=lambda p: (-p[0], p[1]))
print([doc_id for _s, doc_id in scored])
# Output:
# ['a', 'b']

Common mistakes & gotchas

Ranking before filtering

If you score everything first and only then drop the wrong-namespace docs, you've done expensive similarity work on documents you were always going to throw away — and in a real ANN index you can't easily filter after the fact. Filter first, score the survivors. It's both correct and cheaper.

Forgetting the tie-break

scored.sort(key=lambda p: -p[0]) sorts by score alone, so two equal-score docs come back in whatever order they happened to be in. That makes tests flaky and results non-reproducible. Add a secondary key (p[1], the id) so equal scores always order the same way.

Using == on floats for the metadata match

Metadata filters here compare exact values (a namespace string, an int year), which is fine. But never ==-compare the *similarity scores* or any computed float for equality — floating-point rounding means 0.1 + 0.2 != 0.3. Compare scores with < / > for ranking, and use a tolerance if you must test closeness.

Why it matters

Retrieval quality makes or breaks a RAG system, and most retrieval bugs aren't bad embeddings — they're missing filters returning confidently-wrong context (last year's price, another tenant's data). Designing the store as filter-then-rank, with deterministic ordering, is the difference between a demo and something you can trust in production.