Meet the New AI Infrastructure Powering the Next Gen of AI Applications

We’ve been investing in AI-first software for more than a decade at Emergence, exemplified by our early investment in Chorus.ai (sold in 2021 for $575M). The early AI-first companies built their AI tech stacks from scratch. Today, as powerful generative AI foundational models have been made publicly available, new companies are adopting AI into their existing workflows and software applications faster than ever before. Companies, including several in our portfolio like Doximity, Ironclad, and Saleo, have been experimenting with these foundational models and LLMs across numerous facets of their operations, from deploying AI copilots to using AI to power key internal workflows and customer interactions. 

As enterprises become more innovative (and sophisticated) with their use of LLMs, they are realizing that they want more ownership and control over the AI models they use. Enterprises can’t rely solely on closed models like OpenAI’s GPT-4, which are black boxes with no defensibility. It’s our belief that the most sophisticated enterprises will leverage tens if not hundreds of AI models simultaneously, including proprietary models they pre-train from scratch, open source models fine-tuned to their specific needs, and closed models. To facilitate the surge in enterprise AI adoption, we need a new AI infrastructure stack that is focused on speed, cost-effectiveness, transparency, security, and control of your proprietary data/IP. 

Building the foundation for generative AI 

Companies that want to build AI-powered applications and workflows need access to three things (all of which are incredibly limited today):

• AI knowledge: The experience and knowledge required to pre-train or fine-tune foundational models to build unique AI products. Right now, there simply aren’t that many AI researchers with the necessary breadth and depth of model-building knowledge. More are becoming fluent in AI tech everyday as the industry develops, but there is a gap in supply and demand.
• Compute: The GPUs necessary to train AI models. Companies that provide access to GPUs have limited availability due to capacity constraints from GPU manufacturers. This will not be a problem forever as the production scales with time and access increases, but right now, the limited amount of compute is a critical challenge for the overall industry to solve in order to keep up with the rising enterprise demand for AI infrastructure. 
• AI Tooling: The software that enables companies to transform raw data into useful AI models. In order to train or tune a foundational model, enterprises need to clean and tag data, structure that data into a database, and then run algorithms to pre-train a model with that data. Once a model is pre-trained, in order to get value, an enterprise needs to host that model, run inference queries against it, and use the result in an application or workflow. Today, this lifecycle is siloed and mostly manual at each step.

Each of these three elements is essential to developing AI-powered enterprise applications, however today, they are also bottlenecks. As a result, only a fraction of software companies are truly capable of leveraging proprietary AI models. Once AI infrastructure becomes widely available and tooling matures, we believe all software companies will incorporate generative AI into workflows and applications. 

Ultimately, we believe there will be more winners created in the AI software application layer, and these winners will come in multiple flavors. Whether these apps are replacing human-powered services, creating AI copilots that sit alongside business users, or creating brand new workflows with gen AI at the core, we are excited to invest in the Cambrian explosion that this emerging AI infrastructure stack will unleash. 

New AI infrastructure needs to enable enterprises to use multiple foundational models in concert

Right now, companies can choose between leveraging three different types of AI models, each with their own trade-offs: 

• Closed models: OpenAI’s GPT-4 and similar models are ideal for non-customized use cases. They’re cost-effective for rapid experimentation but have limitations on model control, security, and transparency.
• Open models: These models can be fine-tuned with industry-specific or proprietary data and augmented with RAG techniques to better suit a company’s specific needs. These models are the most cost-effective in production, and they reduce vendor lock-in, which increases control over future costs. 
• Proprietary models: These are reserved for high-value, highly customized use cases. They are the most expensive and time-consuming to develop, but they offer a competitive edge through proprietary data and intellectual property control. 

Ultimately, we believe that the most sophisticated enterprises will leverage tens if not hundreds of AI models simultaneously, including all three types of models outlined above. The companies that will succeed in this new AI world will be the ones that best match AI models to their specific use cases, and every software company will need the tooling to train, tune, and orchestrate all these models. 

To this end, we are excited to announce our investment in a company at the heart of this emerging AI infrastructure stack that directly addresses this need. Together.ai is a platform that provides enterprises with everything they need to build their own foundational AI models or leverage open-source ones. You can read more about our investment here. If you’re building in this area or have thoughts on the future of AI infrastructure, I’d love to hear from you; please reach out at joe@emcap.com.