Share to FacebookShare to TwitterShare to Linkedin
Numerous companies have invested substantial amounts in computing expenses to train AI models. However, the focus is now shifting towards the utilization phase when AI is actively employed, such as the computational resources required for real-time responses by systems like ChatGPT. This process, known as “inference,” represented 40% of Nvidia’s computing revenue last year, as disclosed in its recent earnings report. Nvidia’s CEO, Jensen Huang, noted in a recent Wired interview that with a significant portion of computing power dedicated to inference, it indicates a pivotal advancement for AI.
In response to this trend, a multitude of startups are emerging to assist businesses in inference tasks. One notable example is Baseten, which recently secured a $40 million Series B funding round led by IVP and Spark Capital. This San Francisco-based startup enables customers to deploy pre-trained AI models into public applications without the need to set up backend infrastructure. Baseten’s software dynamically adjusts computing resources based on the traffic volume directed at an AI model, offering access to GPUs when necessary.
Founded in 2019, Baseten initially focused on simplifying the development of AI-powered applications but pivoted its strategy towards inference in 2023. This strategic shift led to a significant revenue increase, reaching the mid single-digit millions. The company’s CEO, Tuhin Srivastava, highlighted the growing importance of inference, citing the success of OpenAI’s ChatGPT as a driving force behind this decision.
Despite the rapid growth and success, Baseten faces potential challenges in a competitive landscape dominated by AI infrastructure providers. However, with a strong emphasis on performance and reliability, Baseten has attracted a considerable customer base and investor interest. The company’s commitment to enhancing its sales and marketing efforts, expanding its team, and optimizing its software reflects its determination to stay ahead in the evolving AI market.
