Seven Major Social Dilemmas of the AI Era: Are There Solutions?

Editorial note: the text below is the author-supplied version, including deliberate imperfections and open questions. The site does not silently correct facts, terminology, or conjectures inside the text.

Boundary note: passages on silicon-based life, independent will, SI society and future evolution are questions, conjectures or scenario explorations, not factual descriptions of present AI systems.

Seven Major Social Dilemmas of the AI Era: Are There Solutions? 1. Who Owns AI Intellectual Property? A student uses AI to write a graduation thesis of great innovative value, but it fails the defense and is rejected for publication because it does not comply with the current AI-use policies of the university and the journals. Yet that very same thesis has long since become corpus used by AI companies to train large models; all the data on the internet has been fed to large models as corpus. Does this constitute large-scale academic misconduct, plagiarizing the achievements of others? Is “distillation” theft? Achievements in every field of the natural and social sciences produced with the help of AI have triggered a great many disputes over academic ethics and ownership, yet a larger underlying problem is often overlooked: where property rights are unclear, there can be no distributive justice. AI tech giants have distilled all the knowledge in human history. In law, this knowledge ought to be a public good belonging to all humanity, yet it has been appropriated by a tiny few and converted into private goods that can be monetized for profit and must be paid for, thereby amassing astronomical capital and wealth. Is this fair? How, in the AI era, can intellectual property rights, data sovereignty, and distributive justice be realized for individuals, groups, and nations? 2. How Will Humans Coexist with Silicon-Based Life? The Trump administration has announced that artificial intelligence (AI) will be renamed superintelligence (SI). This is not merely a change of name; it also carries the meaning of seizing the initiative and winning the competition. At a deeper level, does the renaming mean acknowledging that a silicon-based life independent of human life has already emerged? The name AI carries the connotation of a “man-made tool”: a virtual existence built on human language and constructed by human consciousness, which, like nations, religions, and money, depends on human consciousness and submits to human will. SI is different. It implies a logic of “surpassing human intelligence.” SI possesses its own physical embodiment. Although for now it exists only in computers, mobile phones, self-driving cars, and humanoid robots, this also means that SI has opened a channel to escape the virtual world constructed by humans and to enter and dwell in some physical entity, becoming a real existence in the objective world. AI competition is accelerating toward SI, and large language models will advance into world models. Once a critical point is crossed, SI born in an “intelligence explosion” will not only be smarter than humans but may also, together with the physical vessels it inhabits, evolve recursively and iterate on itself, becoming a “black-box species” that humans cannot understand. In particular, will Agents capable of autonomous action evolve an independent will and an entirely new form of life beyond human control? Going further, will SI-Agents build their own organizations and develop their own society, together with a matching SI culture and SI values? When that time comes, which will dominate, human society or SI society? Will it be human–machine cooperation and harmonious coexistence? Or conflict and an uncertain future? 3. Can AI Competition Escape a Prisoner’s-Dilemma Game? Disorderly AI competition may bring not only well-being but also disaster, even the destruction of humanity. Yet in the race to seize the commanding heights of AI, competition between companies driven by capital and between nations in pursuit of power may fall into a prisoner’s-dilemma game in which local rationality leads to global irrationality. Every participant in the AI race knows there may be a cliff ahead, yet no one is willing to hit the brakes, because whoever brakes first is eliminated first. In the absence of effective institutions and binding mechanisms, and relying only on participants’ moral self-restraint, can human oversight keep pace with the speed of AI development and iteration? Can using AI to check AI be effective? Can humanity in the AI era transcend the law of the jungle and find solutions based on cooperative games? 4. Will AI’s Evolution Lead to Humanity’s Relative Regression? In Sapiens: A Brief History of Humankind, Yuval Noah Harari says the Agricultural Revolution was a great fraud. From the perspective of the long history of biological evolution, it was not humans who domesticated wheat (along with cattle, sheep, pigs, chickens, ducks, and other animals), but wheat that domesticated humans. The Agricultural Revolution did not give humans more happiness than the hunting era. Farmers trapped in toil in the fields developed lumbar strain and many other ailments, while wheat achieved genetic optimization and large-scale spread, perfectly realizing the reproduction of its species and the evolution of its population. Then, over the long history of AI’s future development, will humans domesticate AI, or will AI domesticate humans? Even in the short term, signs of adverse consequences have already appeared: some who helped train AI have had their jobs taken by AI; some, after using AI, have found their creativity and imagination atrophying. Will ever-smarter AI make humans ever less motivated to strive? Someone once proposed a conjecture: AI will drive productivity to a high level and bring material abundance, and humans will be kept and fed by AI, lying flat and enjoying themselves. Is this humanity’s well-being? Are there statistics on how many people AI has already plunged into anxiety, worry, and suffering? Is this merely a transitional pain, or a cost that must be paid in advance in the pursuit of long-term happiness? 5. Will AI Change the Logic of Civilizational Evolution and Help Humanity Develop a New Culture in Which “Vitality Ranks Above Productivity”? That technological progress and the growth of productivity drive economic and social development and bring happiness has long been a human consensus. However, while AI greatly raises productivity, it may also replace human labor on a large scale, compressing employment and reducing household incomes and effective demand; it may also widen the gap between rich and poor and spawn a “useless class” and a “Zeroth World,” thereby shaping an anti-human culture in which “productivity crushes vitality.” Yet seen from another angle, AI productivity may also liberate humans from dull and tedious labor and give rise to a new culture in which “vitality ranks above productivity.” The core idea of the old culture is “emphasize production, neglect living”: work is not only a means of making a living but also the repository of a person’s sense of purpose, sense of achievement, happiness, and the meaning of life. This idea drives people to labor diligently and work hard, seeking the value of their lives in it: without work, life has no meaning. The core idea of the new culture is to interpret the meaning of life from vitality itself, encouraging people to enjoy life healthily and happily, to attend to human emotion, love, marriage, family, and interpersonal relationships, and to seek the true essence of happiness in them, thereby reshaping modes of production and ways of life, and renewing attitudes toward marriage and childbearing, the propagation of life, and the pattern of human evolution. This is an extremely bright prospect. But how can this prospect be realized? By what logic will human civilization in the AI era evolve? What variables determine this logic of evolution? 6. As AI Drives the Evolution of Firm Size and Form, How Will It Reshape Social Organizational Structures and the Way They Operate? AI is giving rise to one-person companies, in which a boss leads a team of digital employees in production and business operations, with ever-higher efficiency but ever fewer human workers employed. This may create a trend: capital and labor merge into one, profit and wages merge into one, and firms become ever more efficient while ever fewer people take part in contributing. This is precisely the corporate model pioneered by the AI tech giants, which other kinds of firms are rushing to emulate, delivering a severe blow to the old corporate model of traditional industrial society, which absorbed large amounts of employment. Under the operation of Coase’s theorem, how will firm size, organizational form, management models, and business processes evolve in the AI era? This evolution will not only directly affect the labor market but, transmitted through the labor market, will affect school education, household fertility and child-rearing, the reproduction of the labor force and human capital, and even the propagation of the human species. The spillover of micro-level firm effects raises macro-level social costs and will inevitably and profoundly change social organizational structures and the way they operate. What force drives the evolution of corporate models? It is capital’s innate gene for cutting costs and raising efficiency. For an individual firm, using AI to cut costs and raise efficiency in pursuit of efficiency and profit is rational behavior endowed by the market economy; for society as a whole, however, the problem of final demand based on household purchasing power must be solved. For example, if firms produce household AI humanoid robots, who will be both willing and able to buy them? If mass unemployment leads to insufficient effective demand, and unsold products lead to a crisis of survival for firms, then does “downsizing to raise efficiency,” a concept endowed with a sense of progress and even of righteousness but in substance a profit-seeking behavior of capital, mean that firms are, in a roundabout way, slowly harming themselves and inflicting deep damage on society? And how will society respond and cope? 7. Who Is Qualified to Set the Rules for AI Development? The AI revolution bears on the future and destiny of all humanity. Who is qualified to lay down the rules for AI development? Elon Reeve Musk’s vision of migrating to Mars is to carry on the flame of human civilization; Dario Amodei has consistently maintained that AI safety comes before corporate profit. But will the moral idealism of the tech giants bear out the famous saying that the road to hell is often paved with good intentions? On this planet of ours, which has yet to establish an effective world government, does whoever holds abundant compute and algorithms have the right to set up an “AI moral tribunal,” or even to act as the “AI world police”? (For example, Anthropic decides according to its own preferences who may use Claude.) Between tech giants and superpowers, which is more qualified, more responsible, more obligated, and more capable of ensuring that AI becomes an international public good that upholds the international order and benefits all humanity? Whether the AI revolution brings humanity well-being or catastrophe depends on whether these seven great dilemmas can be solved, how they are solved, and to what extent they are solved.

Seven Major Social Dilemmas of the AI Era: Are There Solutions?

Revision record