Regardless of the public’s opinion on the matter, be it positive or negative, there is little doubt that Artificial Intelligence (AI) has been one of the most transformative technological advancements of the modern era. Its rapid evolution has sparked conversations across industries, from automation to ethical considerations. At the heart of this revolution are Large Language Models (LLMs), which have gained significant traction in recent years. While their capabilities continue to expand, so do the challenges associated with their use.

AI has been particularly significant in the field of technology, where it has revolutionised everything from software development to data processing and decision-making. Understanding how and why developers use AI is crucial to assessing both its benefits and limitations. Whether it’s assisting in code generation, automating repetitive tasks, or improving debugging processes, AI has become an indispensable tool for many in the tech industry.

For Cleverbit, this is a crucial topic. For years, we’ve been dedicated to finding the most effective solutions to add value for our clients, and AI plays a key role in that mission. Efficiency is our North Star, guiding not just our client solutions but also our internal processes and development workflows. That’s why we set out to explore how our own developers perceive the benefits and challenges of AI.

The Rise of Modern AI: A Brief History

The AI revolution did not start with ChatGPT’s viral moment. LLMs like GPT had been in development for years before they reached today’s widespread adoption. While the idea of an AI assistant that could seemingly “know it all” had been envisioned long before, it was Google that pioneered the concept of a ubiquitous AI-powered assistant. However, OpenAI took this vision further by implementing an intuitive interface to GPT, positioning it as an accessible and practical tool for everyday use.

This pattern – attention, funding, advancements, repeat – is what’s driven the rapid evolution of AI. Each technological leap captures public and corporate interest, leading to further investment and subsequent improvements. However, with the exponential growth of their capabilities, the conversation around AI has slowly shifted from “can we build this?” to “how do we use it responsibly?”.

What Does an LLM Actually Do?

One of the main criticisms aimed at LLMs like ChatGPT is that, at its core, they’re essentially a prediction engine. And yes, that is true. Given a sequence of words, LLMs predict the next most probable word based on the context provided. However, while this may seem like a simple process, LLMs are remarkably effective at producing coherent and contextually relevant text.

The evolution of these models from simple text prediction engines to sophisticated processing systems has enabled them to generate human-like responses, summarise vast amounts of information, and assist in complex tasks like programming and content creation.

The Limitations of LLMs

However, condensing such a vast and complex problem (understanding and generating human language) into a predictive framework comes with inherent issues. One of the most discussed challenges is hallucinations – instances where the model confidently generates incorrect or misleading information.

For users unfamiliar with AI’s limitations, these hallucinations can lead to misinformation and flawed decision-making.

One notable case involved Air Canada’s AI chatbot, which misled a customer into purchasing a full-price ticket under the false impression that he could retroactively apply for a bereavement fare. Remarkably, the airline argued that the AI was “responsible for its own actions” and, therefore, they were not liable for compensation. However, the legal system ultimately ruled against Air Canada, setting a significant precedent for the risks associated with AI-generated misinformation.

The Role of Humans in the AI Workflow

Ultimately, while AI can augment human capabilities, it is not a replacement for human skill. The concept of human-in-the-loop AI ensures that AI remains a tool rather than an unchecked authority.

Humans alone can achieve a baseline of 100% accuracy in tasks they are skilled at. Conversely, AI, on its own, operates at a fraction of that percentage. However, when used together, the sum is greater than its parts. Instead of replacing developers, AI serves as an enhancement, freeing them to focus on higher-value tasks.

AI adoption at Cleverbit: Insights from Our Developer Survey

We wanted to understand how our developers perceive and utilise AI in their daily work. We conducted an internal survey in line with the Stack Overflow 2024 AI survey so we could compare industry-wide trends with our internal adoption rates.