Vector Databases Explained: How AI Stores and Searches Embeddings

Modern AI applications increasingly work with embeddings.

An embedding converts information such as text, images, or audio into a numerical vector that represents useful patterns and semantic relationships.

For example:

"How do I reset my password?"

can be converted into a vector such as:

[0.12, -0.43, 0.71, 0.08, ...]

If an application has thousands or millions of these vectors, it needs an efficient way to store them and find the vectors most similar to a user's query.

This is where vector databases become important.

A vector database is a database system designed to store, index, and search high-dimensional vectors efficiently.

In this article, we will explore:

What vector databases are
Why traditional databases are different
How vectors are stored
Similarity search
Distance metrics
Vector indexes
PostgreSQL with pgvector
Python examples
Semantic search
Metadata filtering
RAG systems
Hybrid search
Vector database architecture
Performance considerations
Practical limitations

What Is a Vector Database?

A vector database stores numerical vectors and provides efficient similarity search.

A simplified record might look like:

Document:
"Python is a programming language."

Embedding:
[0.12, -0.43, 0.71, 0.08, ...]

Instead of searching only for exact words, the database can search for vectors that are close to a query vector.

The basic workflow is:

User Query
    ↓
Embedding Model
    ↓
Query Vector
    ↓
Vector Database
    ↓
Similarity Search
    ↓
Relevant Results

This is the foundation of many AI-powered search systems.

Why Do We Need Vector Databases?

Traditional databases are excellent for structured queries.

For example:

SELECT *
FROM products
WHERE category = 'laptop';

This searches for an exact value.

But imagine asking:

"Find products similar to a lightweight computer
for programming."

The database needs to understand semantic relationships.

An embedding model can convert the query into a vector:

Query
 ↓
Embedding Model
 ↓
[0.21, -0.14, 0.72, ...]

The vector database then searches for nearby vectors.

This is called vector similarity search.

Traditional Database vs Vector Database

A traditional relational database stores structured information such as:

ID
Name
Price
Category
Date

A vector database stores information such as:

ID
Text
Embedding
Metadata

The two approaches solve different problems.

Traditional Search

"python tutorial"

may look for matching words.

Vector Search

"How can I start learning Python?"

can find content such as:

"Beginner's guide to Python programming"

even though the wording is different.

What Is a Vector?

A vector is simply a sequence of numbers.

For example:

[0.12, 0.45, -0.32]

Real embeddings are usually much larger:

[
    0.123,
   -0.452,
    0.781,
    0.091,
    ...
]

The number of values is called the vector dimension.

For example:

VECTOR(384)

means the vector contains 384 numerical values.

The dimension depends on the embedding model.

How Vector Search Works

Suppose we have three document embeddings:

Document A → [0.1, 0.2, 0.7]
Document B → [0.8, 0.1, 0.1]
Document C → [0.2, 0.3, 0.6]

The user query becomes:

Query → [0.15, 0.25, 0.65]

The vector database compares the query against the stored vectors.

Conceptually:

Query
  |
  +---- Document A → Similar
  |
  +---- Document B → Less Similar
  |
  +---- Document C → Very Similar

The database returns the closest vectors.

Similarity and Distance

Vector databases use mathematical metrics to determine how close vectors are.

Common metrics include:

Cosine similarity
Euclidean distance
Dot product
Manhattan distance in some specialized applications

The appropriate metric depends on the embedding model and use case.

Cosine Similarity

Cosine similarity measures the angle between two vectors.

The formula is:

cosine_similarity(A, B)
=
(A · B) / (||A|| × ||B||)

Conceptually:

Similarity close to 1
        ↓
Very similar direction

Similarity close to 0
        ↓
Less similar

Cosine similarity is widely used for text embeddings.

Euclidean Distance

Euclidean distance measures the straight-line distance between vectors.

For two vectors:

A = [a1, a2, a3]

B = [b1, b2, b3]

the distance is:

√((a1-b1)² + (a2-b2)² + (a3-b3)²)

Smaller distance generally means the vectors are closer.

Dot Product

The dot product is:

A · B

For:

A = [1, 2, 3]

B = [4, 5, 6]

we get:

1×4 + 2×5 + 3×6

which equals:

Some embedding systems are designed to work efficiently with dot-product similarity.

What Is a Vector Index?

Searching every vector individually becomes expensive when a database contains millions of vectors.

A simple approach would be:

Query
 ↓
Compare with Vector 1
 ↓
Compare with Vector 2
 ↓
Compare with Vector 3
 ↓
...
 ↓
Compare with Vector 1,000,000

This is called a brute-force or exact search approach.

It can become expensive at large scale.

Vector indexes improve search performance by organizing vectors so that nearby candidates can be found more efficiently.

Approximate Nearest Neighbor Search

Many vector databases use Approximate Nearest Neighbor (ANN) algorithms.

Instead of guaranteeing an exhaustive comparison against every vector, ANN methods efficiently search for highly similar candidates.

Common indexing approaches include:

HNSW
IVF
Product Quantization
Disk-based ANN techniques

The exact implementation varies between databases.

The trade-off is generally:

More search accuracy
        ↕
More computation

versus:

Faster search
        ↕
Potentially approximate results

HNSW

One popular vector indexing technique is Hierarchical Navigable Small World, commonly abbreviated as HNSW.

HNSW organizes vectors into graph-like layers.

Conceptually:

          Layer 2
       A -------- B
        \        /
         \      /
          \    /
           C

          Layer 1
      A -- B -- C -- D -- E

The search can move through the graph rather than comparing against every vector.

HNSW is widely used for approximate nearest-neighbor search.

Vector Database Architecture

A simplified vector database looks like:

                Application
                     |
                     ↓
              Search Request
                     |
                     ↓
          ┌────────────────────┐
          │   Vector Database  │
          └────────────────────┘
                     |
        ┌────────────┼────────────┐
        ↓            ↓            ↓
    Vector Data   Metadata     Index
        |
        ↓
Similarity Search
        |
        ↓
Top-K Results

The vector database manages both vector storage and efficient retrieval.

PostgreSQL + pgvector

You do not always need a separate specialized vector database.

PostgreSQL can support vector search through the pgvector extension.

This is especially useful when your application already uses PostgreSQL.

The architecture becomes:

Application
    ↓
PostgreSQL
    ├── Normal relational data
    ├── Metadata
    └── Embeddings

This can simplify application architecture.

Creating a Vector Table

With pgvector installed, a table can be created like:

CREATE EXTENSION IF NOT EXISTS vector;

Then:

CREATE TABLE documents (
    id BIGSERIAL PRIMARY KEY,
    content TEXT NOT NULL,
    embedding VECTOR(384)
);

Here:

VECTOR(384)

means that each embedding must contain 384 dimensions.

The dimension must match the embedding model being used.

Insert a Vector

A vector can be stored alongside document content.

INSERT INTO documents (
    content,
    embedding
)
VALUES (
    'Python is a popular programming language.',
    '[0.12, -0.43, 0.71, ...]'
);

In production applications, embeddings are normally generated programmatically rather than manually entered.

Generate Embeddings With Python

Install:

pip install sentence-transformers

Then:

from sentence_transformers import SentenceTransformer

model = SentenceTransformer(
    "all-MiniLM-L6-v2"
)

text = "Python is a programming language."

embedding = model.encode(text)

print(embedding)
print("Dimensions:", len(embedding))

The generated vector can then be stored in a vector database.

Store Embeddings With Python

For PostgreSQL, you can use a database driver such as psycopg.

Install:

pip install psycopg

A simplified example:

import psycopg
from sentence_transformers import SentenceTransformer

model = SentenceTransformer(
    "all-MiniLM-L6-v2"
)

text = "Python is a programming language."

embedding = model.encode(text).tolist()

connection = psycopg.connect(
    "postgresql://user:password@localhost/mydb"
)

with connection.cursor() as cursor:

    cursor.execute(
        """
        INSERT INTO documents (
            content,
            embedding
        )
        VALUES (%s, %s)
        """,
        (text, embedding)
    )

connection.commit()

connection.close()

In a real application, use environment variables for database credentials rather than putting passwords directly into source code.

Searching a Vector Database

Suppose the user asks:

How can I learn Python?

The query is converted into an embedding.

Then PostgreSQL can search for nearby vectors.

With pgvector, a query can look conceptually like:

SELECT
    id,
    content
FROM documents
ORDER BY embedding <=> '[QUERY_VECTOR]'
LIMIT 5;

The <=> operator is used by pgvector for cosine distance.

The exact operator should match the distance metric and index you have chosen.

What Does Top-K Mean?

Vector searches commonly return the top K results.

For example:

LIMIT 5

means:

Return the 5 best matching results.

The workflow is:

Query
 ↓
Vector Search
 ↓
Rank Results
 ↓
Top 5

For example:

1. Python beginner guide
2. Python programming tutorial
3. Introduction to Python
4. Python syntax reference
5. Python development guide

Metadata Filtering

Vector search becomes more useful when combined with metadata.

Imagine storing articles with:

title
category
language
date
embedding

A search can be restricted to a category.

For example:

SELECT
    id,
    content
FROM documents
WHERE category = 'programming'
ORDER BY embedding <=> '[QUERY_VECTOR]'
LIMIT 5;

This combines:

Metadata Filtering
+
Vector Similarity

Why Metadata Matters

Imagine an e-commerce website containing:

10 million products

A user searches:

"lightweight laptop for programming"

The application could filter by:

category = laptop
price < 1500
availability = true

and then perform semantic vector search among the remaining products.

This can improve both relevance and efficiency.

Semantic Search With a Vector Database

A complete semantic search system looks like:

                  Documents
                      ↓
                  Chunking
                      ↓
                Embedding Model
                      ↓
                Vector Database
                      ↓
                 Vector Index
                      ↓
                  User Query
                      ↓
                Query Embedding
                      ↓
                 Similarity Search
                      ↓
                  Top-K Results

This architecture can power AI search systems.

Vector Databases and RAG

Vector databases are especially important for Retrieval-Augmented Generation, or RAG.

RAG allows an LLM to retrieve relevant information before generating an answer.

The architecture is:

                    Documents
                       ↓
                    Chunking
                       ↓
                   Embeddings
                       ↓
                Vector Database
                       ↓
                  Stored Vectors


User Question
      ↓
Question Embedding
      ↓
Vector Search
      ↓
Relevant Documents
      ↓
Prompt + Retrieved Context
      ↓
LLM
      ↓
Generated Answer

The vector database acts as the retrieval layer.

Example RAG System

Suppose a company has:

Employee Handbook
Product Documentation
Technical Manuals
Support Articles
Company Policies

A user asks:

How many vacation days do employees receive?

The system does not need to send every document to the LLM.

Instead:

Question
 ↓
Embedding
 ↓
Vector Search
 ↓
Vacation Policy
 ↓
LLM
 ↓
Answer

This can reduce unnecessary context and improve retrieval.

Chunking Documents

Large documents are usually divided into smaller chunks before creating embeddings.

For example:

Large PDF
    ↓
Page extraction
    ↓
Paragraph splitting
    ↓
Chunk 1
Chunk 2
Chunk 3
...

Each chunk receives an embedding.

A record may contain:

{
    "content": "Employees receive 20 days...",
    "embedding": [...],
    "document_id": 42,
    "page": 15
}

Metadata makes it possible to identify where the retrieved information came from.

Example Chunking Code

A simple token-based chunker can be created like this:

def chunk_text(text, chunk_size=500):

    words = text.split()

    chunks = []

    for i in range(
        0,
        len(words),
        chunk_size
    ):

        chunk = " ".join(
            words[i:i + chunk_size]
        )

        chunks.append(chunk)

    return chunks


text = """
This is a large document containing
many paragraphs and sections.
"""

chunks = chunk_text(text)

for chunk in chunks:

    print(chunk)
    print("---")

Production systems often use smarter chunking strategies based on paragraphs, sentences, headings, token counts, or semantic boundaries.

Storing Metadata

A useful vector database record might contain:

id
document_id
title
content
embedding
category
author
page_number
created_at

For example:

document = {
    "document_id": 1001,
    "title": "Python Tutorial",
    "content": "Python is...",
    "category": "programming",
    "page_number": 5,
    "embedding": embedding
}

This information can be returned together with search results.

Vector Search in Python

A simple in-memory vector search can be built before introducing a database.

from sentence_transformers import SentenceTransformer
from sklearn.metrics.pairwise import cosine_similarity

model = SentenceTransformer(
    "all-MiniLM-L6-v2"
)

documents = [
    "Python is a programming language.",
    "FastAPI is a framework for APIs.",
    "PostgreSQL is a relational database.",
    "React is used for web interfaces."
]

document_embeddings = model.encode(
    documents
)

query = "How can I build a Python API?"

query_embedding = model.encode(
    [query]
)

scores = cosine_similarity(
    query_embedding,
    document_embeddings
)[0]

results = sorted(
    zip(documents, scores),
    key=lambda x: x[1],
    reverse=True
)

for document, score in results:

    print(
        f"{score:.4f} - {document}"
    )

This demonstrates the basic idea behind vector database search.

A production database adds persistent storage, indexes, filtering, concurrency, and scalable retrieval.

Exact Search vs Approximate Search

There are two broad approaches to vector search.

Exact Search

Compare the query against every stored vector.

Query
 ↓
Vector 1
Vector 2
Vector 3
...
Vector N

Advantages:

Exact
Simple
Good for smaller datasets

Disadvantages:

Can become expensive at very large scale

Approximate Search

Use an index to efficiently find likely nearest neighbors.

Query
 ↓
Vector Index
 ↓
Candidate Vectors
 ↓
Top Results

Advantages:

Much faster at large scale

Disadvantages:

Results may be approximate
Index configuration affects performance

Choosing a Vector Database

Popular options include:

PostgreSQL + pgvector
Pinecone
Qdrant
Weaviate
Milvus
Chroma

The best option depends on your project.

For a small application already using PostgreSQL, pgvector can be an attractive choice.

For specialized large-scale vector workloads, dedicated vector databases may provide more specialized capabilities.

PostgreSQL vs Dedicated Vector Database

PostgreSQL + pgvector

Advantages:

Existing PostgreSQL ecosystem
Relational data and vectors in one database
SQL queries
Metadata filtering
Convenient for many applications

Potential limitation:

Very large vector workloads may require specialized architecture and tuning

Dedicated Vector Database

Advantages:

Designed specifically around vector workloads
Specialized indexing and retrieval
Can provide scalable vector-search infrastructure

Potential limitation:

Adds another service to your architecture
May increase operational complexity and cost

There is no universal winner.

Hybrid Search

Vector search is powerful, but keyword search is still useful.

Consider a query containing an exact product code:

"Product ABC-123"

A keyword search may be extremely useful because the exact identifier matters.

A semantic search system may focus more on meaning.

A hybrid search system combines both.

                 User Query
                     ↓
            ┌────────┴────────┐
            ↓                 ↓
      Keyword Search    Vector Search
            ↓                 ↓
            └────────┬────────┘
                     ↓
               Result Ranking
                     ↓
                Final Results

This can provide stronger retrieval for many real-world applications.

Reranking

A vector database may return the top 20 candidate documents.

A reranking model can then evaluate those candidates more carefully.

User Query
    ↓
Vector Search
    ↓
Top 20 Candidates
    ↓
Reranker
    ↓
Top 5 Results

This two-stage architecture is common in sophisticated search systems.

Vector Database Performance

Performance depends on several factors:

Number of vectors
Vector dimensions
Index type
Distance metric
Hardware
Filtering
Query volume
Database configuration
Number of returned results

For example:

10,000 vectors

may be easy to search with brute force.

But:

100,000,000 vectors

requires much more careful architecture.

Memory Usage

Vectors can consume significant storage.

Suppose you have:

1,000,000 vectors

and each vector has:

1,536 dimensions

If each dimension is stored as a 32-bit floating-point number:

1,536 × 4 bytes
=
6,144 bytes

per vector, ignoring additional index and metadata overhead.

For one million vectors:

≈ 6.14 GB

just for the raw vector values.

Indexes and metadata require additional storage.

This demonstrates why vector storage needs to be planned carefully.

Reducing Vector Storage

Some systems use techniques such as:

Quantization
Lower-precision data types
Product quantization
Dimensionality reduction

These can reduce storage and improve performance.

However, reducing precision or dimensionality can affect retrieval quality.

There is usually a trade-off between:

Storage
Performance
Accuracy

Vector Database Security

Vector databases should be treated like any other production database.

Important considerations include:

Authentication
Authorization
Encryption
Network security
Access control
Tenant isolation
Backup
Monitoring

This becomes especially important when embeddings contain information derived from private documents.

An embedding should not automatically be treated as harmless just because it is represented as numbers.

Multi-Tenant Vector Databases

SaaS applications may have multiple customers.

For example:

Company A
 ├── Documents
 └── Embeddings

Company B
 ├── Documents
 └── Embeddings

A search request from Company A must never accidentally retrieve Company B's data.

Metadata filtering can help:

SELECT *
FROM documents
WHERE tenant_id = 123
ORDER BY embedding <=> '[QUERY_VECTOR]'
LIMIT 5;

Proper authorization must still be enforced at the application and database levels.

Common Vector Database Mistakes

Mistake 1: Using the Wrong Embedding Dimension

If the model produces 384 dimensions:

VECTOR(384)

must be used.

Do not mix vectors from incompatible models without a deliberate strategy.

Mistake 2: Ignoring Metadata

Only storing:

embedding

makes it difficult to understand where results came from.

Store useful metadata such as:

document_id
title
category
page
source

Mistake 3: Poor Chunking

Huge chunks may contain too much unrelated information.

Tiny chunks may lose important context.

Chunk size should be tested against your data and retrieval task.

Mistake 4: Returning Too Many Results

Retrieving 100 irrelevant documents does not necessarily improve an LLM response.

Start with a reasonable top-K value and evaluate retrieval quality.

Mistake 5: Treating Similarity as Truth

A high similarity score means the vectors are close according to the chosen metric.

It does not guarantee that the retrieved content is correct.

Practical Vector Search Project

Let's build a small knowledge-search system.

Install:

pip install sentence-transformers scikit-learn

Create:

vector_search.py

Then:

from sentence_transformers import SentenceTransformer
from sklearn.metrics.pairwise import cosine_similarity

model = SentenceTransformer(
    "all-MiniLM-L6-v2"
)

documents = [
    {
        "title": "Python",
        "content": "Python is a popular programming language."
    },
    {
        "title": "FastAPI",
        "content": "FastAPI is a framework for building APIs with Python."
    },
    {
        "title": "PostgreSQL",
        "content": "PostgreSQL is an open-source relational database."
    },
    {
        "title": "React",
        "content": "React is a JavaScript library for building user interfaces."
    }
]

texts = [
    document["content"]
    for document in documents
]

embeddings = model.encode(
    texts
)

query = input(
    "Enter your question: "
)

query_embedding = model.encode(
    [query]
)

scores = cosine_similarity(
    query_embedding,
    embeddings
)[0]

results = sorted(
    zip(documents, scores),
    key=lambda x: x[1],
    reverse=True
)

print("\nSearch Results:\n")

for document, score in results[:3]:

    print(
        f"Score: {score:.4f}"
    )

    print(
        f"Title: {document['title']}"
    )

    print(
        f"Content: {document['content']}"
    )

    print()

Try:

How can I build an API using Python?

The FastAPI document should be among the highest-ranked results.

From Prototype to Production

The previous example stores vectors in memory.

A production system can replace the in-memory list with:

PostgreSQL + pgvector

or another vector database.

The architecture becomes:

              Web / Mobile App
                     ↓
                  Backend
                     ↓
              Embedding Model
                     ↓
             Vector Database
              ↙           ↘
        Vector Index     Metadata
              ↓
         Search Results
              ↓
              LLM
              ↓
           Response

This architecture can scale far beyond a simple Python script.

Complete AI Search Architecture

A production AI knowledge platform might use:

                    ┌───────────────┐
                    │   Documents   │
                    └───────┬───────┘
                            ↓
                       Text Parser
                            ↓
                        Chunking
                            ↓
                    Embedding Model
                            ↓
                   ┌────────────────┐
                   │ Vector Database│
                   └───────┬────────┘
                           │
                           │
User → Query → Embedding → Search
                           ↓
                    Retrieved Chunks
                           ↓
                        Reranker
                           ↓
                     Context Builder
                           ↓
                           LLM
                           ↓
                       AI Response

This architecture is commonly used for enterprise search, documentation assistants, customer-support systems, and RAG applications.

When Should You Use a Vector Database?

A vector database is useful when your application needs to:

Search by semantic meaning
Store large numbers of embeddings
Find similar documents
Build RAG systems
Build recommendation systems
Search images or multimedia
Cluster related content
Perform similarity matching

You may not need one for a simple application containing only a few hundred vectors.

For small datasets, an in-memory index or a regular database extension may be sufficient.

Final Summary

Vector databases provide the storage and search infrastructure required for many modern AI applications.

The basic process is:

Text
 ↓
Embedding Model
 ↓
Vector
 ↓
Vector Database
 ↓
Vector Index
 ↓
Similarity Search
 ↓
Relevant Results

For RAG:

Documents
 ↓
Chunking
 ↓
Embeddings
 ↓
Vector Database
 ↓
User Question
 ↓
Query Embedding
 ↓
Similarity Search
 ↓
Relevant Context
 ↓
LLM
 ↓
Answer

The most important concepts to remember are:

Vector
    ↓
Numerical representation of data

Vector Database
    ↓
Database designed to store and search vectors

Similarity Search
    ↓
Find vectors close to a query vector

Vector Index
    ↓
Makes large-scale vector search faster

Metadata
    ↓
Additional information used for filtering and results

RAG
    ↓
Retrieval + Context + LLM

Hybrid Search
    ↓
Keyword Search + Vector Search

Vector databases are an important part of the modern AI stack. They provide the connection between embeddings and useful applications, allowing developers to build semantic search engines, document assistants, recommendation systems, AI knowledge bases, and RAG-powered applications.

Once you understand embeddings, vector similarity, indexing, and retrieval, you have the foundation needed to build many of today's practical AI search systems.

Muhit AI — Build the Future, Together.