This post is part of the Premium offering: our Premium subscribers can submit their posts to be published on Turing Post. Send us your stories. Everything from the modern tech world will be considered.

Artificial General Intelligence (AGI) has been a tantalizing goal for AI researchers for decades, embodying the vision of machines that can not only mimic but match or even exceed human cognitive capabilities across a wide array of tasks. As technology advances, the possibility of achieving AGI feels closer, yet its definition remains frustratingly elusive. In my paper, The Frustrating Quest to Define AGI, I explore the ambiguity surrounding AGI’s definition, reviewing various approaches from industry leaders and researchers and suggesting that much of the challenge lies in our understanding of intelligence itself.

AGI has garnered significant attention with the rise of Large Language Models (LLMs) like GPT, which can engage in complex conversations, generate creative content, and perform various intellectual tasks. However, as my paper illustrates, these advances, while impressive, do not constitute AGI as originally envisioned. There is still a broad gap between the sophisticated, yet narrow, capabilities of current AI systems and the versatile, adaptable intelligence that AGI promises.

Defining AGI: The Challenge of Human-Level Intelligence

At its core, AGI aims to create machines capable of understanding, learning, and performing any intellectual task that a human can, and doing so with a versatility that rivals or surpasses human intelligence. Yet, as many researchers note, intelligence is difficult to define, let alone general intelligence that spans various domains. Psychologists and AI researchers alike have reached a consensus that trying to pin down the essence of intelligence is a dead end. This complicates the task of defining AGI – if we cannot define intelligence in the first place, how can we measure its general form?

The classical definition of AGI, often referenced in AI literature, is articulated by Russell and Norvig in Artificial Intelligence: A Modern Approach. They define AGI as systems that can "understand, learn, and perform any intellectual task that a human being can." This definition, though widely cited, remains open to interpretation. For instance, what does it mean for an AI to understand? Is mimicking human responses sufficient, or does true understanding involve something deeper, such as subjective experience or consciousness? Additionally, what constitutes an "intellectual task"? These are the kinds of questions that my paper delves into, showing that even the most authoritative definitions of AGI leave much to be desired in terms of clarity.

Frameworks and Definitions from Industry Leaders: Google and OpenAI

In an attempt to provide structure to the concept of AGI, major players in the AI industry, such as Google DeepMind and OpenAI, have put forth their own frameworks for AGI. These frameworks, while helpful in advancing the conversation, also reflect the diverse interpretations of what AGI could or should be.

Google DeepMind, for example, has developed a five-level model that attempts to describe the progression of intelligence from human equivalence to superhuman capabilities. In their hierarchy, AGI begins at level one, where machines perform at a level comparable to an unskilled human, and ends at level five, where machines outperform all humans in every cognitive domain. This framing is appealing in its simplicity but also leaves critical questions unanswered. As pointed out in the paper, terms like "unskilled," "skilled," and "expert" are not clearly defined within this model. Are we measuring a machine's ability to perform specific tasks, or are we evaluating its competence in broader cognitive domains? Furthermore, intelligence, particularly human intelligence, does not increase linearly. It follows a bell-curve distribution, where the distance between moderate and high intelligence is not the same as the leap from high to genius-level intelligence. The linear progression proposed by Google does not adequately account for this complexity.

Similarly, OpenAI has created a five-level framework that describes AGI's development in terms of increasingly sophisticated capabilities, starting with the ability to interact conversationally and ending with machines capable of performing the work of an entire organization. OpenAI's model focuses more explicitly on LLMs' potential to evolve into more autonomous systems, eventually becoming agents that can solve problems, innovate, and coordinate complex tasks. However, I criticize this framework for its disproportionate scaling between levels. The jump from conversational interaction to problem-solving at the level of a PhD holder, for example, is far more significant than the shift from expert-level problem-solving to organizational management.

Both Google’s and OpenAI’s models represent an effort to create tangible benchmarks for AGI development, but they fall short of providing a comprehensive view of what AGI truly entails. As my paper suggests, these frameworks may be more reflective of each organization’s internal roadmaps and commercial interests than an objective, universally accepted definition of AGI.

The “Cheapening” of AGI: Redefining Success?

One of the more provocative ideas introduced by the paper is the notion that the AGI definition is being "cheapened" by certain leaders in the tech industry. Nvidia CEO Jensen Huang, for example, suggested that AGI could be defined simply as the ability of a machine to pass any test placed in front of it. In this view, AGI could be achievable within five years if we lower the bar to this more attainable standard.

I find this approach deeply flawed – and honestly, quite specious! – so I am arguing that passing tests does not equate to genuine task performance or intelligence. I use the analogy of a cleaning agent to illustrate his point: just because a liquid can clean a surface does not mean it understands the broader process of cleaning a house, which involves multiple interconnected tasks. Likewise, an AI might excel at passing certain benchmarks or standardized tests, but this does not mean it possesses the kind of general intelligence that AGI aspires to.

This reductionist view risks oversimplifying the complexity of AGI and misrepresenting the challenge it presents. By setting such narrow criteria for success, the tech community may prematurely declare victory in the quest for AGI, without achieving the true goal of creating machines that can think, learn, and adapt across all domains of human intelligence.

Turing Test and Social Acceptance: Will We Know It When We See It?

The Turing Test has long been considered a significant milestone in the development of AI. Created by Alan Turing in 1950, the test evaluates a machine's ability to exhibit intelligent behavior indistinguishable from that of a human. Yet the Turing Test was never intended to be a practical gauge for AGI. Instead, it was a thought experiment that has since taken on a life of its own in the public imagination.

Melanie Mitchell of the Santa Fe Institute argues that the Turing Test is problematic because it relies on deception – if a machine can trick a human into believing it is also human, it passes the test. However, this does not necessarily mean that the machine possesses true intelligence or understanding. My paper builds on this argument, suggesting that the real measure of AGI may be something more akin to U.S. Supreme Court Justice Potter Stewart's famous remark about obscenity: "I know it when I see it." In other words, AGI may not have a clear, objective definition, but rather will be recognized through social consensus.

I propose that AGI will be declared when society at large – especially influential critics –agrees that machines have reached a level of intelligence comparable to humans. This could happen gradually as AI systems become more integrated into everyday life and take on increasingly complex tasks. Just as we have come to accept autonomous vehicles and AI-driven customer service agents, so too will we eventually accept AGI, even if it does not arrive in the dramatic, transformative way we initially imagined.

Human Perception and the “Jagged Frontier” of Intelligence

One of the more compelling points in my paper is the discussion of the “jagged frontier” of AI capabilities. Today’s AI systems, such as LLMs, can perform certain tasks with superhuman precision while failing spectacularly at others. This creates a disjointed picture of AI intelligence, where machines seem both incredibly powerful and laughably limited, depending on the task at hand.

This uneven performance complicates the human perception of AI. We tend to expect that smarter systems will make fewer mistakes, but AI does not follow the same cognitive patterns as humans. An AI might solve a complex mathematical problem while failing at a simple logical puzzle. This inconsistency challenges our ability to assess AI’s true capabilities, as we are used to thinking of intelligence as a more unified, consistent trait.

The paper also highlights the feedback loop between AI improvement and human expectation. As AI systems get better, our expectations rise accordingly. Early chatbots like ELIZA were once hailed as breakthroughs, but today they would be considered laughably simplistic. This constant escalation of expectations means that AGI, when it arrives, might not feel as revolutionary as we expect. It will likely be the result of incremental improvements rather than a sudden leap in machine intelligence.

A Matrix of Capabilities: A New Framework for AGI

In the face of these challenges, we may need to rethink how we conceptualize AGI. Instead of viewing it as a linear progression of increasingly complex tasks, I propose a hierarchical matrix of capabilities. This framework would assess AI's proficiency in a wide range of domains, from intellectual and social tasks to physical and emotional ones.

By breaking intelligence down into specific capabilities and measuring AI's performance across different domains, we can gain a more nuanced understanding of its strengths and weaknesses. For example, AI might excel at tasks that require logical reasoning but struggle with tasks that demand emotional intelligence or physical dexterity. This matrix approach would allow us to track AI's progress more accurately, without relying on a singular, all-encompassing definition of AGI.

Conclusion: AGI as a Social Construct

Ultimately, it might be that the quest to define AGI may be less about technical achievement and more about social acceptance. AGI will not be recognized by a single breakthrough or milestone, but by a gradual shift in perception as AI systems become more capable and integrated into society. As humans, we may never agree on a precise definition of intelligence – whether human or artificial – but we will likely come to accept AGI when we see it, even if it does not match the futuristic visions we once held.

In the end, the frustrating quest to define AGI reflects the complexities of intelligence itself, and perhaps the most significant breakthrough will be our own ability to recognize and adapt to a world where machines share in our cognitive labor.