OpenAI details the storage behind 1 billion ChatGPT users: Cosmos DB at unmatched scale, reads 12x faster

• OpenAI published part II of its Habitat online-storage engineering series on October 23, 2026 — the follow-up to its September 11 post on turning Habitat from a Python library into a service.
• Scale: Habitat handles 70M+ requests per second and 500+ petabytes of data across ~40 geographic regions, serving products used by 1B+ people each week (ChatGPT, the API, Codex, internal services).
• Largest Cosmos DB account: tens of petabytes, 20M+ queries per second — the largest Cosmos DB deployment OpenAI knows of, per the company.
• Performance: median global read time cut from ~120ms to ~10ms — a 12x improvement since January; more than 90% of Habitat requests are reads.
• Reliability journey: single shared container per account → separate containers per large service → shared containers by default with data mobility; an improved Cosmos DB serverless mode ended manual container management — no provisioning or noisy-neighbor incidents since.
• Next: OpenAI is developing its own exabyte-scale storage system for Habitat's bulkiest data, pairing Rockset's RocksDB Cloud core with Raft-based replication.

OpenAI has published part two of its engineering deep-dive into Habitat, the storage platform behind ChatGPT: a Cosmos DB deployment larger than any other the company knows of, two outages it learned from the hard way, and median global read times cut from roughly 120 milliseconds to 10 since January.

Tens of petabytes, 20 million queries a second

Habitat is the online storage platform OpenAI built so its products — ChatGPT, the API, Codex and internal services — can access data fast and reliably. Per OpenAI's first installment, published September 11, it now handles more than 70 million requests every second, stores over 500 petabytes, and spans almost 40 geographic regions for products used by over 1 billion people each week. This second post goes a layer deeper: Azure Cosmos DB, Microsoft's cloud database, which sits underneath it all.

OpenAI's largest Cosmos DB account stores tens of petabytes and handles more than 20 million queries per second — and the company says that, as far as it knows, no other company has deployed Cosmos DB this large. Getting there was a three-phase journey: first a single shared container in each account, then separate containers for each large service or feature, and finally a return to shared containers by default, with the ability to move data between them. The Cosmos DB team built an improved serverless mode for OpenAI that ended manual container watching and expansion; since then, OpenAI says it has had no provisioning incidents and no "noisy neighbour" incidents — workloads hogging shared resources.

Two outages, told honestly

The post is unusually candid about what broke. In late 2024, a power cut in the main Azure region of one of its largest Cosmos DB accounts revealed that moving the biggest accounts to a secondary region would take several hours — something neither OpenAI nor the Cosmos DB team had ever tried at that size. Around a month later, while moving those accounts back to the original region, they caused another serious outage. Today, the Habitat and Azure teams monitor every layer together, from OpenAI's services down to Cosmos DB's physical storage.

The 12x read-speed push

More than 90% of requests to Habitat are reads, and speeding them up has been this year's priority. The levers: keeping data close to users when capacity allows, a cache, and the Python-to-Rust rewrite of the service itself. Altogether, the median read time worldwide dropped from roughly 120 milliseconds to roughly 10 — a 12-fold improvement since the performance push began in January. The slowest requests still have to fetch data that isn't in the cache, often from distant regions. (Blockchain.News and IA en un minuto both covered the post the same day.)

Part of the speed story is what Habitat refuses to do. Its storage model is deliberately constrained: objects and edges, each object colocated with its outgoing edges so the data partitions cleanly, with no arbitrary SQL and no general graph traversals. Complex queries are pushed to an offline secondary view — change data capture feeds isolated Rockset instances — keeping the online path fast and predictable.

Next: an exabyte-scale system of its own

For the next 1 billion users, OpenAI says it is developing its own exabyte-scale storage system, designed for Habitat's bulkiest data — pairing Rockset's RocksDB Cloud core with Raft-based replication. The series makes one thing clear: at this scale, storage is the product.