Image Generated with Nightcafe

A Case for Operational-Centric AI

Proposing a model for understanding the evolution of AI from the perspective of engineering resource investment

Artificial Intelligence has been on a relentless march of progress, a fact often masked by the outward simplicity of tools like ChatGPT. The genesis of AI, rich and complex, encourages us to look forward and ask, what is next? Enter the “DMO” Model — a strategic mental model that interprets AI’s evolution through the Data, Model Development, and Operations (DMO) trichotomy.

Proposing the AI Engineering Evolution — “DMO” Model

The “DMO” Model (Figure 1) distinguishes itself by framing the AI evolution across four distinct modern epochs, each characterized by its predominant engineering focus and resource allocation. Let’s take a moment to review these epochs.

Figure 1. Visual representation of the AI Engineering Evolution “DMO” Model. The four plots show a qualitative representation of investment into three key components of the ML lifecycle, Data, Model Development, and Operations. — Image by Author

Epoch #1: The Traditional ML Era

In the early 2010s, machine learning was in an era of aggressive experimentation (Figure 2). Various algorithms from support vector machines to random forests, were employed with an unwavering pioneering spirit. Sparse and unrefined data played a secondary role in the trial-and-error of model architectures, laying bare the nascent state of data engineering and quality control.

Figure 2. The plot shows a qualitative representation of the resources invested into each of the DMO components. The Traditional ML Era shows high investment in Model Development and low investments in the Data and Operational components. — Image by Author

Epoch #2: Neural Net Mania

By the mid to late 2010s, the landscape had shifted. The renaissance of neural networks, bolstered by deeper architectures and larger datasets, set a new bar (Figure 3). The field matured to appreciate the intricacies of data — its acquisition, cleansing, and feature engineering. Concurrently, model optimization focused on hyperparameter tuning and exploring novel regularization techniques to combat overfitting in data-hungry deep neural networks.

Figure 3. The plot shows a qualitative representation of the resources invested into each of the DMO components. The Neural Net Mania Era shows high investment in Model Development, increased investment in Data, and low investments into the Operational Component. — Image by Author

Epoch #3: The GenAI Revolution

Our current epoch, catalyzed by the introduction of transformer architectures, has brought forward the GenAI Revolution (Figure 4). Here, significant investment in model pre-training has begun to dissipate in favor of open-source foundational models. Focus has shifted toward fine-tuning, leveraging proprietary data — creating the “data is your moat” paradigm to create custom GenAI tools. This era underlined the significance of dataset quality, augmentation techniques, and the synthesis of generative models, which has begun to take precedence over exhaustive architectural engineering and “training from scratch”.

Figure 4. The plot shows a qualitative representation of the resources invested into each of the DMO components. The GenAI Revolution Era shows increased investment into Data, a significant decrease in Model Development, and an increase in the Operational component. — Image by Author

Epoch #4: Democratized AI Era

The Democratized AI Era, which could be in full swing as early as 2025, would be mainly characterized by reaching the pinnacle of open-source model performance (Figure 6). In this future scenario, foundational models comparable to GPT-4 would become universally accessible, governed by permissive licenses like Apache 2.0, thereby unlocking their total research and commercial potential.

This progress aligns with an imminent plateau in data accessibility and quality (Figure 5). The reservoir of novel, human-generated content needed to endow models with new emergent abilities is not infinite. As we approach this threshold, the focus will shift to refining the curation and enhancement of existing datasets until we also encounter the boundaries of this endeavor.

Figure 5. This simplistic qualitative diagram depicts the trajectory of model performance as it intersects and surpasses the specific limits. (1) The first limit defined by the blue line shows the limits of transformer technology (2) The second orange line shows the limits defined by data scarcity (3) shows model performance increasing through innovations in software and hardware. There is an undefined theoretical limit of software and hardware signaled by arrows at the top of the image. — Image by Author

Upon encountering this data scarcity ceiling, the emphasis will transition to software and hardware innovations that facilitate efficient model fine-tuning and optimization. These advancements aim to tailor these models to specific applications while minimizing the computational footprint required for training and inference. Today’s vanguard reflects such innovations: parameter-efficient tuning (PEFT) methodologies like LoRA (Low-Rank Adaptation) and QLoRA (Quantized Low-Rank Adaptation), alongside model compression strategies such as quantization and sparsity techniques already integrated into mainstream frameworks. Furthermore, the evolution of CPU, GPU, and ASIC deep learning performance will continue a relentless march toward performance per watt efficiency with every generation.

Even though theoretical ceilings in software and hardware enhancements may loom, it’s probable that we’ll encounter a level of performance where the entry barriers are significantly lower prior to hitting these ceilings. This threshold will be characterized by the trivialization of tasks like fine-tuning, making them so cost-effective and rapid that costly cloud compute resources are no longer a prerequisite for crafting practical AI-driven solutions.

Once the dual milestones of peak transformer technology performance and minimal barriers to entry are secured through hardware and software optimizations, the landscape will starkly change unless we stumble into a trove of human-generated data or pivot to a radically different technological paradigm beyond transformers.

Upon breaching this juncture, we enter the era of truly Democratized AI, where the ubiquity of AI parallels the commonplace utility of tools like Microsoft Excel. In this context, the discerning factor for competitive advantage is not the tool itself but how it is leveraged. This is the quintessence of technological democratization: intensified operational competition. Reflect on Airbnb’s disruption of the hospitality sector; once the marketplace allowed everyone to operate their own “hotel,” the differentiation lay in the operational ingenuity — reimagined guest experiences, streamlined check-in processes, and enhanced customer service.

In the case of AI, democratization will funnel significant resources into optimizing Operational AI, often called MLOps. This shift brings critical questions to the forefront: How will organizations integrate these powerful tools seamlessly into their existing systems? What strategies will they employ to ensure these tools can scale effectively with demand, and how will they guarantee safety and compliance? Moreover, in a landscape where advanced AI tools are widely accessible, the challenge will be to innovate novel business models that leverage this technology to create value and sustain a competitive edge.

Figure 6. The plot shows a qualitative representation of the resources invested into each of the DMO components. The GenAI Revolution Era shows decreased investment into Data, a significant decrease in investment in Model Development, and sharply increased investment in the Operational component. — Image by Author

Concluding Thoughts on the Dawn of Operational-Centric AI

We have already witnessed an evolution from Model-Centric to Data-Centric AI, and now we are on the cusp of an Operational-Centric AI paradigm. As AI becomes more pervasive and democratically accessible, its true potential unfolds — not merely as a tool, but as a catalyst for innovation and societal benefit.

The Operational-Centric AI future is not simply a theoretical construct but a tangible destination towards which the industry is inexorably moving. The “DMO” model proposes that success in this new era will be achieved by those who can harness Operational AI to deliver transformative solutions, affirming once again that the future belongs to those who are prepared to innovate not just in technology but in how technology is operationalized within the fabric of society.

Disclaimer: This is a personal opinion piece and in no way reflects the strategy or the position of my current employer on the topic discussed.

Don’t forget to follow my profile for more articles like this!