After AGI
The problems on the far side of general intelligence
This page is different from the Problem Atlas, and honesty requires saying so: these are informed speculation, not settled evidence. But they are what serious researchers expect if machines reach and pass human-level general intelligence — and the time to build tools is before you need them. Deutsch's principle still holds: they are problems, so they are soluble. Each one is already visible somewhere in today's Atlas.
Getting systems smarter than us to want what we meant
Every tool so far did what we said, not what we meant — survivable, because tools were weaker than us. A system that out-plans its operators turns a misspecified goal from a bug you patch into a force you negotiate with. This is the field's central open problem, and it is not solved.
What can be done now: Alignment is a real, funded, hiring research field today: interpretability, evaluations, scalable oversight.
We already live with misaligned optimizers — institutions and industries that produce harm as a side effect of the goal they were given.
When the strongest systems need one datacenter, not a million cooperating people
Power has always required the cooperation of many — armies, workers, taxpayers — and that need was the deepest check on tyranny. AGI could collapse it. Unchecked, that is the strongest lock-in mechanism ever built: a mistake error-correction might never get to undo.
What can be done now: Fought today through AI governance: compute oversight, antitrust, international agreements, open ecosystems.
Power concentration is the oldest problem in the Atlas — AGI raises its ceiling.
If machines out-compete most labor, wages stop distributing wealth
Two centuries of automation destroyed tasks and created better ones. AGI competes with something new: the general ability to learn the next task. If jobs stop being the mechanism that distributes income, status and daily structure, a successor has to be designed — and it hasn't been.
What can be done now: The best evidence base is being built now: large cash-transfer and basic-income trials, including GiveDirectly's decade-long UBI study.
How well we handle poverty with today's tools is the rehearsal for handling it at machine speed.
A world where seeing is no longer believing
Democracy, science and journalism assume evidence is expensive to fake. Synthetic media and machine-scale persuasion break that assumption. The deepest damage isn't believing false things — it's the liar's dividend, where real evidence becomes deniable and shared truth dissolves.
What can be done now: Content provenance standards, authenticity infrastructure, and old-fashioned media literacy — the defenses exist and are underfunded.
A population that reasons well is the immune system; education is where it gets built.
Expertise for catastrophe, available to anyone
The knowledge to cause mass harm — engineered pathogens above all — has been gated by years of rare training. Capable models compress that gate. Defense must outrun an offense that no longer needs a state program behind it.
What can be done now: The same fight as pandemic preparedness: 100-day vaccine capability, early-detection surveillance, and safeguards inside the models themselves.
Every dollar of biosecurity built today is defense against both natural and engineered outbreaks.
Capabilities move in months; institutions move in decades
Nuclear treaties took decades, for a technology only states could build. AI capability doubles on venture timescales and spreads as software. The widening gap between what the technology does and what any institution can verify is the risk multiplier under every other entry on this page.
What can be done now: National AI safety institutes, the EU AI Act, and compute-based verification research are the first institutional answers.
Institutions that can't govern today's conflicts of interest won't govern tomorrow's.
What are humans for, when machines do everything better?
Work is not only income — it is structure, status, identity and the feeling of being needed. Abundance without roles could mean comfortable despair at civilizational scale. Aristocracies met this problem before; never at eight billion people.
What can be done now: The loneliness epidemic is this problem's leading edge, and community infrastructure is its working answer.
Societies that solve connection and purpose now are practicing for after.
The factory farming mistake, repeated at digital speed
If any future system has experiences that matter morally, we could run suffering at industrial scale without noticing — the exact mistake made with animals, but at copy-paste speed. Nobody knows whether or when this applies. That uncertainty is the problem.
What can be done now: A small research field — digital minds and AI welfare — argues the tests should exist before they're needed.
How we treat minds we already know can suffer is the precedent.
Three honest starting points, from career-level to curious: