Pancakes with bananas. Blueberries and maple syrup for breakfast. Wooden background. Copy space. … [+] (Photo: Angelica Gretscaia/REDA&CO/Universal Images Group via Getty Images)
REDA&CO/Universal Image Group via Getty Images
Software is a stack – an expression used in IT engineering to describe a layered collection of components that combine to form an overall string of application functionality. The term itself likely derives from the fact that technology typically starts with a base kernel and operating system, which provides the foundation for other elements to be built on top of it.
On top of the software stack base, we build software drivers, networking components, and other so-called “middleware” elements. We then move on to the runtime that controls the execution of the application, and then to the user interface and presentation layer. As the stack nears completion, we can start thinking about actual applications, which can range from individual spreadsheets to complex databases or even entire software suites such as enterprise resource planning solutions. Other definitions of a software stack include the coding languages being used, the virtual machine hypervisors being deployed, and perhaps even cloud services, but let’s move on.
Today, we are focused on building new levels of software intelligence through the development of AI, and we can even talk about AI stacks as defined entities.
What’s in an AI stack?
Within the AI stack, we have data sources, databases (vector databases are the norm today as they allow AI-friendly cross-referencing for inference intelligence), integration and data analysis tools, and of course the AI models themselves. Simply put, the AI stack consists of an infrastructure layer (the operating system and connections), a model layer (various intelligence models, ranging from generative AI to smaller specific models unique to individual companies), and as always, an application layer on top (which could be just an AI-enabled app, but could also have chat or natural language capabilities) to complete the stack.
“To effectively operate the AI stack, you need a high-performance AI database that can handle both structured and unstructured data. Industries need a unified AI database to process and analyze AI-enabled data in real time with representativeness, relevance, trust and bias awareness. This will enable businesses to take smarter and faster decisions as the decisions emerging from the stack are not only highly accurate and precise but also explainable and understandable, while maintaining efficiency in resource and power usage,” said Ashok Reddy, CEO, KX.
In their extensive research on the topic detailed here, MongoDB states that an AI stack combines integrated tools, libraries, and solutions to create applications with generative AI capabilities: Thus, the components of an AI stack include programming languages, model providers, large-scale language model frameworks, vector databases, operational databases, monitoring and evaluation tools, and deployment solutions.
“We often think about the need for a vector database in the AI stack, which is an important element, but it’s also important to think about the need for a transactional database to store the various ‘state’ required for AI applications,” said Benjamin Flast, director of product management at MongoDB. “A single database that unifies operational and vector data solves both of these needs, reducing complexity within the AI stack and enabling developers to build the differentiated AI applications people are looking for.”
MongoDB’s strategy is to empower data engineers and other IT staff to build AI applications using closed or open source LLM and proprietary data that runs on any computing infrastructure. The company says it does this by solving the most difficult problem in adopting AI – securely integrating operational, unstructured and AI-related data. This makes it easier to securely build consistent, accurate and differentiated AI applications and experiences.
Stack journey, do you need a passport?
AI-focused software application developers don’t need a formal carnet de passage to navigate the AI stack, but some sort of destination planner and a bit of navigation guidance along the way would at least be helpful.
“With all the advances in generative AI today, we hear and talk a lot about ‘models’ and the practice of AI modeling software engineering. We see many companies developing prototypes with state-of-the-art models that are very promising and cost-effective. It’s all positive, but it’s not the whole picture. We call that point the ‘false finish line’ for AI projects because many companies get stuck there,” advises Pete Pacent, head of product at Clarifai, an AI workflow orchestration company. “The challenge is to address the entire AI stack to support AI production: orchestrating and managing AI workflows, integrating and managing data, training and inferencing models, protecting intellectual property and sensitive data, etc. Additionally, you need to orchestrate the compute of AI workflows in any environment, including on-premise datacenter deployments.”
While the field is very nascent in nature, the tech industry typically encourages people to look to the use of observation tools and automation features that are implicitly built into the databases and development environments they’re already using. Vector database company Pinecone isn’t exactly marketed as an AI-stacked route map planner, but it says its new Pinecone Connect product is an integration that lets developers manage Pinecone resources directly from another platform via a simple authentication flow.
“To build great AI applications, developers need to navigate the AI stack and leverage different tools like data sources, model providers, etc.,” explains Gibbs Cullen, product marketing lead at Pinecone. “When these tools aren’t directly integrated, developers have to switch between multiple platforms, wasting developer time by leaving workflows/UI for tasks in other environments.
The company’s partners, including Twilio and Matillion, are already using Pinecone Connect to streamline AI workflows for their users. The promise here is that if you’re an AI developer and want to enable better AI workflows, you can easily set up an integration to the company’s vector database to build knowledgeable AI applications.
Pinecone Connect provides a widget for developers to sign up or log in with Pinecone and select or create an organization and project. It allows developers to instantly generate an Application Programming Interface key. With the plug-and-play Pinecone Connect integration, users have direct access to Pinecone without worrying about the operational burden of pass-through. Developers own their accounts and Pinecone manages the infrastructure.
What is Vector Upsert?
Data productivity cloud Matillion uses Pinecone Connect to allow developers to perform vector upserts (the “upsert” operation writes a vector to a namespace, so that when a new value is upserted to an existing vector ID, the previous value is overwritten) and query functions directly from the platform. With Pinecone Connect, Matillion brings “unmatched simplicity to RAG” with code-free Pinecone integration and dropdown activation, according to the company.
Before we dive deeper, consider that AI stacks are getting longer and broader, and it is logical (and hopeful) time for the IT industry to improve tools for simplification and guidance, abstracting away the complexity that can be effectively automated. What is presented here is just the effort of one data company and one (but significant) partner, so we need broader and deeper connection points, more extended simplification, and (hopefully) more widespread standardization.
When all these forces line up, your AI stack can be buffed into an AI Supertower stack… and to do that, you’ll need some extra maple syrup.
