That’s Not a Job for an LLM: The Right Way to Apply AI to Network Operations

Welcome to Heavy Networking. I'm Ethan Banks with Drew Conry-Murray, your cohost who reminds you to keep those fifty ohm terminators on the ends of your coax to prevent the packets from falling out. And if you get that super lame dad joke, you are you're an engineer of a certain age and you found your tribe.

In today's episode, we're getting off the AI hype train to talk about how different artificial intelligence techniques usefully impact network operations.

"I'm tired of AI being claimed to be any sort of a savior or a game changer... The reality lies somewhere in the middle. And the facts are that AI in various forms represents a set of tools that, like with any tool, has use cases, capabilities, and limitations."

AI is a set of technologies that is worth taking seriously, but so much has come at the network engineering community so quickly that it has been hard for us to get a handle on what this tech actually is and how to use it effectively.

Our guest today is Avi Friedman, the founder of Kentik, our sponsor today. Avi's seen AI from both sides—both employing it to get more out of the Kentik product suite and to provide tooling to the network engineering community.

As we start, it's important to share some history about AI and its use in networking. Before the time of LLMs, we have been using AI in different ways.

Historical Overview of AI in Networking

• AI Winters: There have been multiple AI winters, and between every winter, some tools became just tools without the AI label.
• Expert Systems: Even though they were AI-inspired, many techniques have been around for decades without being actively labeled as such.
• Statistics and Machine Learning: We used these to study patterns of telemetry and to assist in capacity planning and DDoS detection.

Kentik drives DDoS detection for major service providers, leveraging ML and basic statistics effectively.

Distinction between ML and LLMs

• Machine Learning (ML): Involves statistical techniques and pattern recognition over data. It focuses on specific tasks like capacity planning and DDoS detection.
• Large Language Models (LLMs): LLMs take a vast amount of textual data, creating highly sophisticated models that simulate knowledge and can assist in various tasks, but still do not truly understand context in a human sense.

Application and Misunderstandings of LLMs

• While LLMs are touted for their reasoning abilities, much of it comes down to statistical predictions rather than actual reasoning.
• This leads to what some term 'hallucinations'—predictions made without a proper understanding of the universe.

"...LLMs have jumped so rapidly to let us assemble our own fleet of AI tools. But trust needs to be verified."

Practical Applications

• When to Use ML: Anything involving mathematics and statistical analysis.
• When to Use LLMs: Primarily for language processing tasks but with careful attention needed to ensure that outputs align with reality.

Conclusion

As AI and its derivatives continue to evolve, it’s crucial for engineers and technicians in networking to differentiate between these technologies to leverage their strengths effectively, whether it’s through traditional ML or newer LLM methodologies.

Avi’s extensive background shows us that while AI can provide powerful tools, it is important to critically assess how and where we apply these technologies in the real world.