What actually occupies memory in a vector index
A vector index stores four things, and the one people compute is usually the smallest surprise. The raw vectors are dimensions times bytes per component: a 1,536-dimension fp32 embedding is 6,144 bytes, and a million of them is 6.14 GB. That much is arithmetic.
The graph is the part that is easy to forget. Approximate nearest-neighbour search at scale is done with HNSW, a navigable small-world graph in which every vector is a node holding a list of neighbour ids. Those lists are stored, and they are not small: at the common setting M = 16, each node keeps up to 2M = 32 four-byte ids, or 128 bytes. Against a 1,536-dimension fp32 vector that is only 2%, so it disappears into rounding. Against a 384-dimension int8 vector it is a third of the record. Quantise aggressively and the graph, not the data, becomes your dominant cost.
The metadata is whatever payload you attach: source id, title, URL, timestamps, tenant id, and any field you filter on. Teams that store the chunk text itself alongside the vector often find the payload is several times the size of the embedding — an 800-token chunk is roughly 3,200 bytes of UTF-8 text, half again the size of a 1,536-dimension int8 vector.
The overhead covers id maps, deleted-record tombstones that persist until compaction, secondary indexes on filter fields, and allocator fragmentation. It is implementation-specific, which is why it is an input here rather than a constant.
The formula, term by term
d · b is the vector itself. b is 4 for fp32, 2 for fp16 or bf16, 1 for int8 scalar quantisation and 0.125 for binary quantisation, where each component collapses to a single bit. Note that quantisation acts only on this term.
8M is the graph. The per-element memory model published with hnswlib gives roughly (d·4 + M·2·4) bytes per element, which is where the coefficient comes from: 2M neighbour ids of 4 bytes each. HNSW is hierarchical, so nodes also appear in higher layers with M links rather than 2M, but the expected number of higher-layer nodes falls geometrically and contributes a small correction. Raising M improves recall and raises both memory and build time; setting it to 0 here models a flat index that stores no graph at all and scans every vector per query.
The (1 + h) factor multiplies everything, including the graph and metadata, because fragmentation and bookkeeping scale with the records rather than with any single component. The replica factor r multiplies again. Note the distinction from sharding: shards divide one copy of the index across nodes and leave the total unchanged, while replicas are full copies and multiply it. A three-shard, two-replica deployment holds six partitions totalling twice the index size.
Everything is reported in GiB, meaning 230 = 1,073,741,824 bytes. Cloud providers usually bill storage in GB of 109 bytes, which is 7.4% larger a unit, so a 5.84 GiB index is 6.27 GB on an invoice. Check which one a quoted limit uses before sizing to it.
Worked example: 5 million vectors at 1,536 dimensions
You are indexing 5,000,000 chunks with a 1,536-dimension model, stored as fp32, in an HNSW index with M = 16, with 256 bytes of metadata per record, a 15% overhead allowance, and one replica.
- Vector bytes. 1,536 × 4 = 6,144 B.
- Graph bytes. 2 × 16 × 4 = 128 B.
- Metadata. 256 B.
- Subtotal per record. 6,144 + 128 + 256 = 6,528 B.
- With overhead. 6,528 × 1.15 = 7,507.2 B per record.
- Total bytes. 7,507.2 × 5,000,000 = 37,536,000,000 B.
- In GiB. 37,536,000,000 ÷ 1,073,741,824 = 34.96 GiB.
- RAM to serve it. 34.96 × 1.25 = 43.70 GiB, using a 25% working allowance as a planning rule of thumb.
Broken down, that is 28.61 GiB of embedding data, 0.60 GiB of graph links, 1.19 GiB of metadata and 4.56 GiB of overhead. The graph is 1.7% of the index here.
Now switch the stored precision to int8 and hold everything else. Vector bytes fall to 1,536, the subtotal becomes 1,536 + 128 + 256 = 1,920 B, with overhead 2,208 B, and the total is 11,040,000,000 B = 10.28 GiB. The vector data alone shrank by exactly 4×, but the index shrank by 34.96 ÷ 10.28 = 3.40×, because the graph, the metadata and the overhead computed on them did not shrink at all. That gap is the reason to size quantisation from the whole record rather than from the vector.
How to read the result and where it stops fitting
Compare the RAM figure against the machine you intend to run on, because HNSW is a memory-resident structure by design: the search walks the graph following pointers, and every hop that lands on disk costs a random read. An index that does not fit in RAM does not degrade gracefully, it falls off a cliff. The 25% working allowance above is a planning rule of thumb covering query buffers, the operating system's own needs, and the transient extra memory a segment merge or compaction requires; it is not a measured constant, and a store that rebuilds segments in place may need considerably more.
When the total outgrows a node, you have three levers and they are not equivalent. Sharding divides the index across machines and keeps everything in memory, at the cost of querying every shard and merging results. Quantisation shrinks the vectors in place, and the worked example shows the whole-index saving is always less than the per-vector saving. Disk-backed indexes keep the graph and a compressed representation resident while the full-precision vectors live on SSD, re-ranking the shortlist with a small number of reads; this is the approach that scales furthest per node, and it trades a little latency for a large amount of memory.
Look at which component dominates before choosing. If metadata is the largest term, none of the three helps and the answer is to move the payload out: keep an id in the vector store and fetch the text from a document store after retrieval. If the graph dominates — which happens with low dimensions and aggressive quantisation — lower M, or use a clustered index such as IVF that stores centroids rather than per-node neighbour lists.
Finally, size for growth. The vector count comes from your chunking decisions, and halving the chunk size roughly doubles it. Work the two together with the RAG chunking calculator before committing to hardware.
GiB per million vectors by dimension and precision
| Dimensions | fp32 | fp16 | int8 | binary |
|---|---|---|---|---|
| 384 | 1.55 | 0.83 | 0.48 | 0.16 |
| 768 | 2.98 | 1.55 | 0.83 | 0.21 |
| 1,024 | 3.93 | 2.03 | 1.07 | 0.24 |
| 1,536 | 5.84 | 2.98 | 1.55 | 0.30 |
| 3,072 | 11.56 | 5.84 | 2.98 | 0.48 |
Read down a column to see the cost of dimensionality, and across a row to see the cost of precision. The binary column flattens because the fixed 128-byte graph dominates once the vector is only 48–384 bytes.
Sizing mistakes that show up in production
- Counting documents instead of chunks. The vector count is the chunk count. A 5,000-document corpus split at 512 tokens is commonly 15,000–20,000 vectors.
- Omitting the graph. At 1,536 fp32 dimensions it is 2% and forgivable; at 384 int8 dimensions it is 25% of the record and changes the machine you need.
- Storing chunk text as metadata without counting it. An 800-token chunk is roughly 3,200 bytes — larger than a 1,536-dimension int8 vector, and it is stored on every replica.
- Confusing shards with replicas. Shards split one copy; replicas multiply it. Only replicas change the total in this calculation.
- Mixing GB and GiB. Cloud storage is usually billed in GB of 10^9 bytes, 7.4% larger a unit than a GiB. A 100 GiB index is 107.4 GB on the invoice.
- Sizing to exactly the index size. Building or rebuilding an index needs transient headroom, and deleted records are not reclaimed until compaction runs.
- Assuming quantisation preserves recall. It usually costs some, and how much depends on your model and your data. Measure recall@k against an exact search on a query sample before shipping it.
How this compares with other index types and where the cost really sits
HNSW is the default in most vector stores because it gives high recall at low query latency, and this calculator assumes it. Two alternatives change the memory profile materially. IVF partitions vectors into clusters around centroids and searches only the nearest few, storing centroids rather than a neighbour list per node — substantially less index overhead, at the cost of recall that depends on how many partitions you probe. Product quantisation compresses each vector into a short code of subspace centroid ids, often 32–64 bytes regardless of dimension, which is a far larger saving than scalar quantisation and a correspondingly larger hit to precision; it is normally paired with a re-ranking pass over full-precision vectors.
Managed vector services price on their own units — storage tiers, pod sizes, read and write units — and rarely expose the bytes directly. The figure this page produces is still the right input, because it tells you which tier your data lands in and whether a quantisation or dimension change would move you down one.
Note also that dimensionality is a choice, not a given. Several current embedding models are trained so their vectors can be truncated to a shorter prefix with a modest quality loss, which turns a 3,072-dimension model into a 1,024-dimension one and cuts the vector bytes by two thirds without re-embedding at a different provider — though it does require a full re-index, whose cost the embedding cost calculator prices.
If you are weighing this against serving a model on the same hardware, note that vector index memory and model weights compete for the same GPU or host RAM. The GPU VRAM requirement calculator and the quantisation memory savings calculator size that side, and the same quantisation logic — a fixed component that does not shrink — applies there too.