The Size Problem Nobody Talks About Until It Bites You

Embedding models produce vectors. The size of those vectors — the number of dimensions — determines how much storage they consume, how fast similarity search runs, and what the memory footprint of your vector index looks like at scale. A 1536-dimension embedding for a million documents occupies roughly 6 GB of float32 storage, which is manageable. At a hundred million documents — a realistic scale for production document retrieval, semantic search over a large codebase, or recommendation system item embeddings — you are looking at 600 GB of dense vectors before indexing overhead. At that point, the embedding dimension is not an academic detail. It is an infrastructure cost.

The obvious response is to reduce dimensionality. The less obvious question is how to do it without destroying the semantic information the embeddings were generated to capture. Two techniques dominate practical use: PCA applied post-hoc to an existing embedding set, and Matryoshka Representation Learning built into the embedding model at training time. They solve the same problem in structurally different ways, and the difference matters more than most teams realise before they are already committed to one approach.

PCA: The Statistician's Scalpel

Principal Component Analysis is a classical dimensionality reduction technique that projects high-dimensional vectors onto a lower-dimensional subspace defined by the directions of greatest variance in the data. Applied to embeddings, it works as a post-processing step: you take a corpus of embeddings, fit a PCA transform to them, and then project both the stored vectors and any new query vectors through the same transform before computing similarity. The dimensionality of the projected vectors is whatever you choose — you can compress 1536 dimensions down to 256 or 64 without retraining anything.

The catch is the corpus dependency. The PCA transform is computed on a specific dataset and optimised for the variance structure of that dataset. If your corpus changes significantly — new document types, a different language distribution, a shift in the topics you are indexing — the transform that was optimal for the original corpus may preserve the wrong structure in the new one. You will not get an error. Recall will simply degrade, quietly, until someone notices that certain query types are returning worse results than they used to. The PCA matrix has a drift problem that is genuinely difficult to debug because the failure mode is a gradual quality regression rather than a hard failure.

Matryoshka Representation Learning: Compression by Design

Matryoshka Representation Learning, introduced by Kusupati et al. and adopted by several major embedding model providers including OpenAI for text-embedding-3, takes a different approach. Rather than compressing embeddings after training, MRL modifies the training objective so that the first N dimensions of every embedding vector are already a self-contained, semantically meaningful representation at lower dimensionality. The name comes from the Russian nesting dolls: each prefix of the full vector is itself a usable embedding.

The practical consequence is that compression requires only truncation. To get a 256-dimension embedding from a 1536-dimension MRL model, you take the first 256 values and discard the rest. No transform matrix needs to be fitted, stored, or updated. No corpus dependency is introduced. The compressed vector is valid immediately for any query without any additional preprocessing. The cost is that you need a model that was trained with MRL — you cannot apply the technique retrospectively to an existing non-MRL model.

The Trade-offs in Practice

For teams starting a new embedding pipeline with an MRL-capable model, the choice is straightforward: use truncation. The operational simplicity is significant — no transform maintenance, no corpus drift risk, no infrastructure for applying PCA to incoming queries at inference time. The quality at truncated dimensions is competitive with PCA compression in most benchmarks and better in scenarios where the corpus distribution is expected to shift over time.

For teams locked to a non-MRL model — whether because of an existing investment, a specific model's quality on their domain, or a fine-tuned model they cannot retrain — PCA remains the practical tool. It works, it is well-understood, and the corpus drift problem is manageable if you build periodic PCA refit into your pipeline. The risk is not that PCA is wrong; it is that teams often do not build the refit mechanism, discover the problem a year later, and have to reason backwards through months of recall degradation to find the cause.

Benchmarking Matters More Than You Think

Published benchmark comparisons between MRL truncation and PCA compression are informative but not portable. MTEB benchmark results measure quality on standardised datasets that may not share the distributional characteristics of your specific corpus. A technique that preserves 98% of retrieval quality on MTEB may preserve 85% on a domain-specific medical or legal corpus where the embedding model was not optimised and the variance structure is different.

The right approach is to benchmark on your own data at the dimensionalities you are actually considering, using the retrieval metrics that matter for your application — whether that is recall at K, mean reciprocal rank, or something more domain-specific. Both techniques should be tested at the target dimension, not just compared at their published optimal settings. The benchmark that matters is the one that predicts your production behaviour.

Which One Should You Use?

If you are choosing an embedding model today and your use case is compatible with a model that supports MRL — which includes the OpenAI text-embedding-3 family, Nomic Embed, and several others — use MRL truncation. The operational advantages are real, the quality is competitive, and you avoid an entire class of maintenance problem that PCA introduces.

If you are working with an existing non-MRL model and need to reduce storage or query costs, use PCA with a refit schedule built into your pipeline from the start. Accept the corpus dependency as a constraint, instrument recall metrics over time so you can detect drift before it becomes an incident, and treat the PCA matrix as infrastructure that needs maintenance rather than a one-time transformation.