OpenAI Launches Foundry, Allowing Customers to Buy Dedicated Compute for AI Models

The AI race heats up as OpenAI launches developer platform for GPT-3.5 and potential GPT-4 model.

OpenAI is launching a new developer platform called Foundry. It allows customers to run OpenAI's newer machine learning models on dedicated capacity. This means that customers can buy compute capacity (essentially, processing power) that is solely dedicated to running their chosen model. This can improve the performance of the model and give customers more control over its configuration.

OpenAI has privately announced a new developer product called Foundry, which enables customers to run OpenAI model inference at scale w/ dedicated capacity.

It also reveals that DV (Davinci; likely GPT-4) will have up to 32k max context length in the public version. 🔥 pic.twitter.com/5KEsWLqPdc
— Travis Fischer (@transitive_bs) February 21, 2023

Foundry is aimed at "cutting-edge customers running larger workloads," according to documentation that has been published by users with early access. Customers will have the ability to monitor their instances with the same tools and dashboards that OpenAI uses to build and optimize its models, and they will also have some level of version control over the models they use.

However, the cost of using Foundry will be high. Running a lightweight version of OpenAI's latest text-generating model, GPT-3.5, will cost $78,000 for a three-month commitment or $264,000 over a one-year commitment. This is because training and running state-of-the-art AI models is an expensive process that requires a lot of processing power and resources.

The launch of Foundry comes as OpenAI is under increasing pressure to turn a profit, following a multibillion-dollar investment from Microsoft. The company reportedly expects to make $200 million in 2023, but this is a small fraction of the investment that has been put into the startup so far. OpenAI has been exploring a variety of ways to monetize its technology, including launching a "pro" version of its ChatGPT chatbot and teaming up with Microsoft to develop Bing Chat.

In summary, OpenAI's new Foundry platform allows customers to buy dedicated compute to run its AI models, giving them more control over the configuration and performance of the models they use. However, the cost of using Foundry is high, reflecting the expense of training and running state-of-the-art AI models.