The artificial intelligence community is embroiled in a debate about its future and whether it has enough scale to create the doctrine of God.

Written by: Mario Gabriele

Translated by: Block unicorn

The Holy War of Artificial Intelligence

I would rather live my life as if there is a God and die to find out there isn’t, than live my life as if there isn’t and die to find out there is. — Blaise Pascal

Religion is a fascinating thing. Perhaps because it is completely unprovable in any direction, or perhaps like my favorite saying: "You cannot argue facts against feelings."

The characteristic of religious belief is that as faith rises, it accelerates in development at an incredible speed, making it almost impossible to doubt the existence of God. How can you doubt a divine presence when the people around you increasingly believe in it? Where is there room for heresy when the world rearranges itself around a doctrine? Where is there space for opposition when temples and cathedrals, laws and norms are all arranged according to a new, unshakeable gospel?

When the Abrahamic religions first emerged and spread across continents, or when Buddhism spread from India throughout Asia, the immense momentum of faith created a self-reinforcing cycle. As more people converted and complex theological systems and rituals were established around these beliefs, it became increasingly difficult to question these fundamental premises. In a sea of credulity, becoming a heretic is not easy. Grand churches, complex religious texts, and thriving monasteries all serve as physical evidence of a divine presence.

But the history of religion also tells us how easily such structures can collapse. As Christianity spread to the Scandinavian Peninsula, the ancient Norse beliefs crumbled in just a few generations. The ancient Egyptian religious system lasted for thousands of years, only to disappear when new, more enduring faiths arose and larger power structures emerged. Even within the same religion, we have seen dramatic splits— the Reformation tore apart Western Christianity, while the Great Schism led to the division of the Eastern and Western churches. These splits often begin with seemingly trivial doctrinal differences, gradually evolving into entirely different belief systems.

Scripture

God is a metaphor that transcends all levels of intellectual thought. It’s that simple. — Joseph Campbell

Simply put, believing in God is religion. Perhaps creating God is no different.

Since its inception, optimistic AI researchers have imagined their work as creationism— the creation of God. In recent years, the explosive development of large language models (LLMs) has further solidified the believers' conviction that we are on a divine path.

It also confirms a blog post written in 2019. Although people outside the field of artificial intelligence only recently became aware of it, Canadian computer scientist Richard Sutton's "Bitter Lesson" has become an increasingly important text in the community, evolving from obscure knowledge into a new, all-encompassing religious foundation.

In 1,113 words (every religion needs a sacred number), Sutton summarizes a technical observation: "The biggest lesson from 70 years of AI research is that using a general approach to computation is ultimately the most effective and a huge advantage." The progress of AI models has benefited from the exponential increase in computational resources, riding the massive wave of Moore's Law. Meanwhile, Sutton points out that much of the work in AI research has focused on optimizing performance through specialized techniques—adding human knowledge or narrow tools. While these optimizations may be helpful in the short term, in Sutton's view, they are ultimately a waste of time and resources, akin to adjusting the fins of a surfboard or trying new wax when a massive wave is coming.

This is the foundation of what we call the "Bitter Religion." It has only one commandment, commonly referred to in the community as the "Scaling Law": exponential growth in computation drives performance; the rest is foolishness.

The Bitter Religion has expanded from large language models (LLMs) to world models and is now rapidly spreading through biology, chemistry, and embodied intelligence (robotics and autonomous vehicles), which remain untransformed sanctuaries.

However, as Sutton's doctrine spreads, definitions also begin to change. This is a hallmark of all active and vibrant religions—debate, extension, commentary. The "Scaling Law" no longer merely means scaling computation (the ark is not just a boat); it now refers to various methods aimed at enhancing transformer and computational performance, with some tricks included.

Now, the canon encompasses attempts to optimize every part of the AI stack, from techniques applied to the core model itself (model merging, mixture of experts (MoE), and knowledge distillation) to generating synthetic data to feed these ever-hungry gods, with a plethora of experiments in between.

Warring Sects

Recently, a question that has arisen in the AI community, tinged with the spirit of holy war, is whether the "Bitter Religion" is still correct.

This week, Harvard University, Stanford University, and the Massachusetts Institute of Technology published a new paper titled "The Scaling Law of Accuracy," sparking this conflict. The paper discusses the end of quantifying technical efficiency gains, a series of techniques that improve AI model performance and greatly benefit the open-source ecosystem. Tim Dettmers, a research scientist at the Allen Institute for AI, outlined its significance in a post below, calling it "the most important paper in a long time." It represents a continuation of the increasingly heated dialogue over the past few weeks and reveals a notable trend: the growing consolidation of two religions.

OpenAI CEO Sam Altman and Anthropic CEO Dario Amodei belong to the same sect. Both confidently state that we will achieve artificial general intelligence (AGI) in about 2-3 years. Altman and Amodei can be said to be the two figures most reliant on the sanctity of the "Bitter Religion." All their incentives lean towards overcommitting, generating the maximum hype to accumulate capital in this game almost entirely dominated by economies of scale. If the scaling law is not the "Alpha and Omega," the beginning and the end, then what do you need $22 billion for?

Former OpenAI chief scientist Ilya Sutskever adheres to a different set of principles. He, along with other researchers (including many from within OpenAI, according to recent leaks), believes that scaling is approaching its limits. This group argues that to maintain progress and bring AGI into the real world, new science and research will inevitably be required.

Sutskever's faction reasonably points out that the ongoing scaling philosophy of the Altman faction is economically unfeasible. As AI researcher Noam Brown asked, "After all, are we really going to train models that cost hundreds of billions or trillions of dollars?" This does not even include the additional tens of billions in inference computing expenses required if we shift computational scaling from training to inference.

But true believers are very familiar with their opponents' arguments. The missionaries at your doorstep can easily counter your hedonistic dilemmas. For Brown and Sutskever, the Sutskever faction points to the possibility of "compute at test time." Unlike the situation so far, "compute at test time" does not rely on larger computation to improve training but instead allocates more resources to execution. When an AI model needs to answer your question or generate a piece of code or text, it can provide more time and computation. This is akin to shifting your focus from studying for a math exam to persuading the teacher to give you an extra hour and allow you to bring a calculator. For many in the ecosystem, this is the new frontier of the "Bitter Religion," as teams are shifting from orthodox pre-training to post-training/inference methods.

It is easy to point out the flaws in other belief systems, criticizing other doctrines without exposing one's own position. So, what is my own belief? First, I believe that the current batch of models will yield very high returns on investment over time. As people learn to circumvent limitations and leverage existing APIs, we will see the emergence and success of truly innovative product experiences. We will move beyond the reification and incremental stages of AI products. We should not view it as "artificial general intelligence" (AGI), as that definition has structural flaws, but rather as "minimum viable intelligence," capable of being customized according to different products and use cases.

As for achieving superintelligent AI (ASI), it requires more structure. Clearer definitions and delineations will help us discuss the trade-offs between the potential economic value and economic costs of each more effectively. For example, AGI may provide economic value to a subset of users (merely a localized belief system), while ASI may exhibit unstoppable compound effects, changing the world, our belief systems, and our social structures. I do not believe that ASI can be achieved solely through scaling transformers; but unfortunately, as some might say, that is just my atheistic belief.

Lost Faith

The AI community cannot resolve this holy war in the short term; there are no facts to bring to bear in this emotional struggle. Instead, we should turn our attention to what the AI's questioning of its faith in the scaling law means. The loss of faith could trigger a chain reaction, extending beyond large language models (LLMs) and affecting all industries and markets.

It must be noted that in most areas of AI/machine learning, we have not yet thoroughly explored the scaling law; there will be more wonders in the future. However, if doubt truly begins to creep in, it will become more difficult for investors and builders to maintain the same high confidence in the ultimate performance state of "early curve" categories like biotechnology and robotics. In other words, if we see large language models start to slow down and deviate from the chosen path, many founders and investors' belief systems in adjacent fields will collapse.

Whether this is fair is another question.

There is a viewpoint that "artificial general intelligence" naturally requires larger scales, and therefore, the "quality" of specialized models should manifest at smaller scales, making it less likely for them to encounter bottlenecks before providing actual value. If a model in a specific domain only ingests a portion of the data and thus only requires a fraction of computational resources to achieve viability, shouldn't it have enough room for improvement? This seems intuitively reasonable, but we repeatedly find that the key often lies elsewhere: including relevant or seemingly irrelevant data often enhances the performance of seemingly unrelated models. For example, including programming data seems to help improve broader reasoning capabilities.

In the long run, the debate about specialized models may be irrelevant. Anyone building ASI (artificial superintelligence) is likely aiming for an entity that can self-replicate and self-improve, possessing the ability to exert infinite creativity across various domains. Holden Karnofsky, a former board member of OpenAI and founder of Open Philanthropy, refers to this creation as "PASTA" (the Process of Automating Scientific and Technological Advancement). Sam Altman's original profit plan seems to rely on similar principles: "Build AGI, then ask it how to get a return." This is apocalyptic AI, the ultimate destiny.

The success of large AI labs like OpenAI and Anthropic has sparked enthusiasm in the capital markets to support similar "OpenAI for X" labs, whose long-term goal is to build "AGI" around their specific vertical industries or fields. This inference of scale decomposition will lead to a paradigm shift away from OpenAI simulations towards product-centric companies—this is a possibility I proposed at Compound's 2023 annual meeting.

Unlike apocalyptic models, these companies must demonstrate a series of advancements. They will be companies built on engineering problems of scale, rather than scientific organizations conducting applied research, with the ultimate goal of building products.

In the realm of science, if you know what you are doing, you shouldn't be doing it. In the realm of engineering, if you don't know what you are doing, you also shouldn't be doing it. — Richard Hamming

Believers are unlikely to lose their sacred faith in the short term. As mentioned earlier, with the surge of religion, they have compiled a script for living and worship, along with a set of heuristics. They have built physical monuments and infrastructure that reinforce their power and wisdom, indicating that they "know what they are doing."

In a recent interview, Sam Altman said the following when discussing AGI (emphasis on "we"):

This is the first time I feel that we really know what to do. There is still a lot of work to be done from now until building an AGI. We know there are some known unknowns, but I think we basically know what to do, and it will take time; it will be difficult, but it is also very exciting.

Judgment

In questioning the "Bitter Religion," scaling skeptics are reckoning with one of the most profound discussions of the past few years. Each of us has engaged in such thoughts in some form. What would happen if we invented God? How quickly would that God appear? What would happen if AGI (artificial general intelligence) truly and irreversibly arose?

Like all unknown and complex topics, we quickly store our specific reactions in our brains: some despair at their impending irrelevance, most anticipate a mix of destruction and prosperity, and a final group expects humanity to do what we do best—continue to seek problems to solve and address the issues we have created, thus achieving pure abundance.

Anyone with significant stakes wants to predict what the world will look like for them if the scaling law holds and AGI arrives in a few years. How will you serve this new God, and how will this new God serve you?

But what if the stagnant gospel drives away the optimists? What if we begin to think that even God might decline? In a previous article titled "Robot FOMO, Scaling Laws, and Technological Predictions," I wrote:

I sometimes wonder what would happen if the scaling law does not hold, whether it would resemble the effects of revenue loss, growth slowdown, and rising interest rates on many technology sectors. I also sometimes think that if the scaling law does hold completely, whether it would resemble the commoditization curves of many other fields' pioneers and their value capture.

"The benefit of capitalism is that, regardless, we will spend a lot of money to find out the answers."

For founders and investors, the question becomes: what happens next? Candidates who could become great product builders in each vertical are gradually becoming known. There will be more such individuals in various industries, but this story has already begun to unfold. Where will new opportunities arise?

If scaling stagnates, I expect to see a wave of closures and mergers. The remaining companies will increasingly shift their focus to engineering, an evolution we should foresee by tracking talent movement. We have already seen some signs that OpenAI is moving in this direction as it increasingly productizes itself. This shift will open up space for the next generation of startups to "take the curve" by relying on innovative applied research and science rather than engineering, attempting to surpass existing enterprises in the pursuit of new paths.

Lessons from Religion

My view on technology is that anything that seems to have obvious compounding effects typically does not last long, and a commonly held belief is that any business that appears to have obvious compounding effects strangely develops at a rate and scale far below expectations.

Early signs of religious schism typically follow predictable patterns, which can serve as a framework to continue tracking the evolution of the "Bitter Religion."

It often begins with the emergence of competing interpretations, whether for capitalist or ideological reasons. In early Christianity, differing views on the divinity of Christ and the nature of the Trinity led to schisms, resulting in starkly different biblical interpretations. In addition to the AI schisms we have already mentioned, there are other emerging fractures. For example, we see a portion of AI researchers rejecting the core orthodox view of transformers and turning to other architectures, such as State Space Models, Mamba, RWKV, Liquid Models, and others. While these are still soft signals, they indicate the budding of heretical thoughts and a willingness to rethink the field from foundational principles.

Over time, the impatient words of prophets can also lead to distrust among people. When the prophecies of religious leaders do not come to fruition, or divine intervention does not arrive as promised, it sows the seeds of doubt.

The Millerite movement predicted that Christ would return in 1844, but when Jesus did not arrive as scheduled, the movement collapsed. In the tech world, we often silently bury failed prophecies and allow our prophets to continue painting optimistic, long-cycle future versions, despite repeatedly missing the predetermined deadlines (hi, Elon). However, without the support of continuously improving raw model performance, the faith in the scaling law may also face a similar collapse.

A corrupt, bloated, or unstable religion is susceptible to the influence of apostates. The Protestant Reformation made progress not only because of Luther's theological views but also because it emerged during a period of decline and turmoil within the Catholic Church. When mainstream institutions show cracks, long-standing "heretical" thoughts can suddenly find fertile ground.

In the field of artificial intelligence, we may focus on smaller models or alternative methods that achieve similar results with less computation or data, such as the work done by various Chinese corporate labs and open-source teams (like Nous Research). Those who break through the limits of biological intelligence and overcome barriers long thought insurmountable may also create a new narrative.

The most direct and timely way to observe the beginning of a transformation is to track the movements of practitioners. Before any formal schism, religious scholars and clergy often privately hold heretical views while appearing compliant in public. The contemporary equivalent may be some AI researchers who outwardly adhere to the scaling law but secretly pursue entirely different approaches, waiting for the right moment to challenge the consensus or leave their labs in search of theoretically broader horizons.

The tricky part about religion and technological orthodoxy is that they often have a part that is correct, just not as universally correct as the most faithful believers think. Just as religion incorporates fundamental human truths into its metaphysical framework, the scaling law clearly describes the reality of neural network learning. The question is whether this reality is as complete and immutable as the current enthusiasm suggests, and whether these religious institutions (AI labs) are flexible and strategic enough to lead the zealots forward together while establishing mechanisms that allow knowledge to spread (chat interfaces and APIs) to disseminate their knowledge.

The Endgame

"Religion is real to the common people, false to the wise, and useful to the rulers." — Lucius Annaeus Seneca

One potentially outdated view of religious institutions is that once they reach a certain scale, they become susceptible to survival motives, much like many human-run organizations, attempting to survive in competition. In the process, they overlook truth and great motives (which are not mutually exclusive).

I once wrote an article about how capital markets become narrative-driven information echo chambers, where incentive mechanisms often perpetuate these narratives. The consensus around the scaling law gives off an ominous similarity— a deeply rooted belief system that is mathematically elegant and extremely useful in coordinating large-scale capital deployment. Like many religious frameworks, it may be more valuable as a coordinating mechanism than as a fundamental truth.

免责声明：本文章仅代表作者个人观点，不代表本平台的立场和观点。本文章仅供信息分享，不构成对任何人的任何投资建议。用户与作者之间的任何争议，与本平台无关。如网页中刊载的文章或图片涉及侵权，请提供相关的权利证明和身份证明发送邮件到support@aicoin.com，本平台相关工作人员将会进行核查。