What Kind of AI Architecture is Needed to Reimagine Every Experience?

As discussed in my previous blog, artificial Intelligence (AI) is not just a technological advancement; it's a transformative force that has the potential to reimagine entire business experiences. But what do the technological underpinnings need to look like to make this a reality? To propel your AI system capabilities to new heights, you need a Compound AI System, which tackles AI tasks using multiple interacting components that combine logically.

We believe that the approach of agentic systems and workflows highlighted by industry expert Andrew Ng in his blog, is key to achieving those goals. Essential design patterns in this framework include reflection, tools, planning, and multi-agent collaboration. Significant improvements in response accuracy have already been observed using  self-reflection, which is also documented well by recent research.

For business applications that demand high predictability and rely on accurate datasets, tools are crucial in enhancing the system capabilities. These tools provide real-time data access within workflows, enabling efficient task completion. Registering and invoking tools such as SQL, API access, specific machine learning (ML) models , and additional AI models significantly enhance overall system intelligence. They allow AI systems to pull data from various business functions, providing cross-functional insights. Considering how users will interact with network, security, and enterprise data, this integration is crucial. Offloading tasks to these tools also provides an effective way to control both Generative AI performance and costs.

An agentic system, composed of multiple autonomous agents that interact and collaborate to achieve complex tasks, is essential for putting all these tools together and creating the proper steps to solve a given problem. At the heart of this system is the planner, a critical component that autonomously decides the sequence of steps necessary to accomplish larger tasks. While we are not yet ready for full autonomy, as Andrej Karpathy strived for at Tesla with autonomous driving, we are certainly heading in that direction. Another significant opportunity lies in interacting with users to continuously improve the aforementioned sequence of steps. This involves defining and refining the processes the system will follow to accomplish tasks more efficiently and predictably. Mastering this aspect will provide a substantial competitive advantage for those who can effectively leverage user interactions to optimize their AI systems

The design pattern of multi-agent collaboration is about mimicking human teamwork through the coordinated efforts of multiple agents. Collaboration among multiple Large Language Model (LLM) agents, whether based on the same LLM or a mix of different LLMs and Collaboration among multiple Large Language Model (LLM) agents, whether based on the same LLM or a mix of different LLMs and task-specific Small Language Models (SLMs), is crucial for achieving outcomes that typically require human teamwork. Along with other tools such as APIs, these agents work together to handle complex tasks more efficiently and innovatively.

This approach mimics human teamwork and brings significant benefits in the enterprise IT networking context. Just like a human team, where individuals have different areas of expertise, AI agents can be specialized for specific tasks. For example, in a network operations center (NOC), one agent can handle network performance monitoring, another can manage security incidents, and a third can process user access requests. This specialization ensures that each aspect of network management is handled by the most appropriate agent. Multiple agents can work on different parts of a problem simultaneously, reducing the time required to complete complex tasks and increasing overall efficiency. When agents collaborate, they can combine their individual strengths to generate more creative and innovative solutions.

Continuing this approach can lead to a new level of intelligence, autonomy, and system capabilities. According to Yu Huang from Roboraction.AI, agents can be classified into five levels:

• Level 1: Emerging - AI that is equal to or slightly better than unskilled humans.
• Level 2: Competent - AI that matches the skills of at least 50% of skilled adults.
• Level 3: Expert - AI that is equal to at least 90% of skilled adults.
• Level 4: Virtuoso - AI performing at the level of the top 1% of skilled adults.
• Level 5: Superhuman - AI that outperforms all humans.

It feels like Level 3 is very much in reach for us.

Many companies have made significant investments in AI. However, those who fail to rearchitect their systems continuously towards new and emerging architectures risk being caught in the "innovator's dilemma" even at this early phase of the technology cycle. Relying on outdated AI methodologies and treating AI as just a technology add-on can prevent these businesses from achieving AI capabilities that truly mimic human collaboration.

As the world of Artificial Intelligence (AI) rapidly evolves, businesses must constantly adapt to maintain their competitive edge. One crucial step in this evolution is developing an AI architecture that extends beyond just using Large Language Models (LLMs) and Generative AI (GenAI).

The cornerstone of an effective AI architecture lies in building a that does not depend solely on LLMs but combines all available technologies in an effective way. Two components—the use of diverse techniques/tools and the ability to work like a human team—are vital for achieving advanced AI capabilities.

 Building an advanced AI architecture for enterprise IT networking requires moving beyond reliance on LLMs and incorporating an agentic system that integrates various tools and techniques to create what the Berkeley Artificial Research Center BAIR calls Compound AI Systems. By focusing on collaboration and the use of diverse technologies, businesses can achieve AI capabilities that go beyond what the existing LLM capabilities suggest. This approach not only enhances efficiency and innovation but also ensures businesses stay ahead in the competitive AI landscape. The future of AI in enterprise IT is bright, and those who embrace these advancements will lead the way in the next era of technological evolution. For more insight into this topic, including the current stage of the technology cycle and agentic systems, check out this article by Matt Wood from AWS. Stay tuned for the next insight on AI from the Office of the CTO at Extreme Networks.