What exactly is an embedding, and why does AI engineering need them?

An embedding is a list of numbers — a vector — that represents a piece of data in a way that captures its meaning, not just its literal content. A sentence, an image, a product description, a snippet of audio: each can be turned into a vector of, say, 768 or 1536 floating-point numbers. The specific numbers don't mean anything on their own. What matters is the relationship between vectors: two pieces of data that mean similar things end up as vectors that sit close together in that high-dimensional space, and two pieces of data that mean very different things end up far apart.

That single property — semantic similarity becomes geometric distance — is the reason embeddings are the foundation underneath almost every modern AI application that isn't pure text generation: search, recommendation, deduplication, clustering, anomaly detection, and retrieval-augmented generation (RAG) all reduce, at some point, to "find the vectors near this vector."

How embeddings are produced

Embeddings come from a model trained specifically to produce them — an embedding model. You don't hand-design the 1536 numbers for a sentence; you pass the sentence through a neural network (usually a transformer encoder) that has learned, from massive amounts of training data, to map inputs with similar meaning to nearby points in vector space.

A minimal example using a hosted embeddings API looks like this:

from openai import OpenAI
 
client = OpenAI()
 
response = client.embeddings.create(
    model="text-embedding-3-small",
    input="The cat sat on the mat.",
)
 
vector = response.data[0].embedding
print(len(vector))   # 1536
print(vector[:5])    # [0.0023, -0.0091, 0.0184, ...]

Run the same call on "A feline rested on the rug." and you'll get a different vector — but one that sits close to the first, because the two sentences mean nearly the same thing despite sharing almost no words. Run it on "The stock market fell sharply today." and the resulting vector will be far away from both. That's the whole mechanism: meaning in, geometry out.

You can do the same thing locally with an open-weight model, which matters if you care about cost, latency, or not sending data to a third party:

from sentence_transformers import SentenceTransformer
 
model = SentenceTransformer("all-MiniLM-L6-v2")
vectors = model.encode([
    "The cat sat on the mat.",
    "A feline rested on the rug.",
    "The stock market fell sharply today.",
])
print(vectors.shape)   # (3, 384)

Notice the dimension is different — 384 here instead of 1536. Every embedding model defines its own vector size, and that size is fixed for the model's lifetime: you can't compare a vector from one model to a vector from another, even if both happen to be the same length by coincidence. Mixing embedding models in the same index is one of the most common early mistakes teams make.

Measuring "closeness"

Once you have vectors, you need a way to quantify how close two of them are. The most common choice is cosine similarity, which measures the angle between two vectors rather than the straight-line distance between them:

import numpy as np
 
def cosine_similarity(a, b):
    return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))
 
similarity = cosine_similarity(vectors[0], vectors[1])
print(similarity)   # something like 0.82 — high similarity

Cosine similarity ranges from -1 to 1, though in practice embeddings from most modern models rarely produce negative values — you'll typically see scores clustering somewhere between 0 and 1, with values above roughly 0.8 indicating strong semantic overlap. The reason cosine is preferred over raw Euclidean distance is that it ignores magnitude and only cares about direction, which turns out to correlate better with semantic similarity for the way these models are trained. Some embedding models (OpenAI's among them) even produce vectors that are pre-normalized to unit length, at which point cosine similarity and dot product become mathematically equivalent — which is why you'll see both used interchangeably in vector database documentation.

Dense vs. sparse: two different ideas of "vector"

Everything above describes dense embeddings — vectors where every dimension holds a meaningful, usually non-zero floating-point value, produced by a neural network. But there's an older, still-relevant alternative: sparse embeddings, where a vector has thousands or tens of thousands of dimensions (often one per vocabulary term), and almost all of them are zero except for the handful of terms actually present in the text.

In production systems this isn't really an either/or choice. A well-built search system frequently combines both — dense vectors to catch conceptual matches a keyword search would miss, sparse vectors to catch exact terms an embedding model might blur together — and merges the two result sets. That combination is usually called hybrid search, and most modern vector databases support it as a first-class feature rather than something you have to stitch together yourself.

Why embeddings need somewhere to live

A single embedding is just an array of floats sitting in memory — not interesting on its own. The interesting part happens at scale: once you have a few hundred thousand or a few million embeddings, "find the ones closest to this query vector" stops being something you can do with a linear scan in reasonable time. That's the entire reason vector databases exist as a distinct category of infrastructure: they build indexes (structures like HNSW graphs or IVF clusters) specifically so that nearest-neighbor search stays fast as the collection grows, trading a small amount of accuracy for a large amount of speed.

That tradeoff — approximate rather than exact nearest-neighbor search — is worth sitting with for a moment, because it surprises people coming from traditional databases. A vector database is very rarely guaranteeing it will return the mathematically closest vectors; it's guaranteeing it will return vectors that are very likely close, computed in a fraction of the time an exhaustive search would take. For almost every real application — search, recommendations, RAG retrieval — that's the correct tradeoff to make, because the difference between the true 5th-nearest neighbor and the actual 6th-nearest result returned is rarely meaningful to the end user, while the difference between a 20-millisecond query and a 4-second one absolutely is.

Practical implications for AI engineers

A few things matter more in practice than the theory above might suggest:

• Pick one embedding model per index and keep it. Because vector spaces from different models aren't comparable, re-embedding your entire dataset is required any time you switch models — there's no partial migration path.
• Store the model name alongside your vectors. It's easy to forget which model produced a given batch of embeddings, especially once a project has been running for a year and had its embedding model upgraded once or twice.
• Normalize inputs before embedding them when it matters. Casing, whitespace, and formatting differences can shift embeddings slightly; for exact-duplicate detection this matters much more than for general semantic search.
• Batch your embedding calls. Nearly every embedding API accepts a list of inputs per request rather than one string at a time — this is dramatically cheaper and faster than looping over individual calls.

Embeddings are the layer underneath nearly everything else this section covers — vector databases, ANN indexing strategies, hybrid search, RAG pipelines. Understanding what they actually are, and what they aren't (a database, a search algorithm, a magic semantic-understanding button), is the foundation the rest of it builds on.