Scaling online storage to serve over 1 billion ChatGPT users (Part II)
OpenAI · · Infrastructure
Part two of OpenAI's Habitat series covers the storage layer under its products: more than 500 petabytes on Azure Cosmos DB, with the largest account serving over 20 million queries per second. The team went from multi-tenant containers to single-tenant and back again, after hot-key throttling, request-tier quality of service, and an improved serverless mode fixed the noisy-neighbor problems that pushed them away the first time.
Why it matters: A rare public look at multi-tenancy tradeoffs at a scale where a managed service stops being hands-off.