|
Share: Link
Aaron Gertler
November 11, 2026
|
|
What’s it like to work with people who truly believe that AGI is coming?
Over the last few years, we’ve been thrilled to hear from thousands of people who are newly worried about AI progress and want to work on our priority problems.
But we’ve also heard from employers we work with that many of these people don’t really “get it.” They have the right technical skills, and they’ve learned the basics of safety work, but — as one of my colleagues said in Slack — they “haven’t internalized the moral or empirical worldview” that informs the strategy of most organisations within the field.
This ‘worldview gap’ is one of the most common reasons that highly qualified people don’t land jobs in AI safety. Here’s why it matters so much.
|
|
Many people in the field hold views I’d sum up like this:
Empirical: There’s a good chance — perhaps greater than 50% — that we’ll live in a radically different world within the next few years. (Maybe three, maybe 10. Not 30.)
Moral: If AI goes badly, we could face human extinction, permanent disempowerment, or other catastrophic outcomes. These are far more important than other AI-related problems, even if the other problems still matter.
It makes sense that these views don’t come naturally to many people. They would have seemed crazy to almost anyone 10 years ago and are still unusual today. And to be clear, the gap isn’t about agreement: you can do good work in AI safety while doubting these premises. But you do need to respect them and think clearly about what they imply, because people around you will be acting on them.
Some of our advisees accept radical change and catastrophic risk as hazy possibilities, but they don’t seem to expect those things to really happen. They treat AI safety the same way they’d treat zoning reform or lead paint mitigation: important and interesting, but not so different from any other problem. It doesn’t dominate their thoughts about their future or change how they plan for it.
Meanwhile, the people who read their applications often see AI safety as a desperate struggle to survive a transition larger than the Industrial Revolution. Some expect AI to kill them (and everyone else). Others think they’ll probably survive, but end up in a world as alien as Manhattan would be to Marcus Aurelius. When they read about AI displaying superhuman ability at hacking or mathematics, they expect most other fields to meet the same fate, producing a world where humans are at risk of obsolescence.
|
|
To get a glimpse of this thinking, see computer scientist Scott Aaronson, who used to be sceptical about AGI but now reports his position as:
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
Or Dean Ball, who used to advise the Trump administration on AI:
We are stepping into a novus ordo seclorum—a new order of the ages […] I just can’t shake the feeling that all of this is going to change in ways I cannot quite define.
Or the many people in AI safety who have warned their loved ones that we may be approaching a strange and dangerous future. See George Rosenfeld of Coefficient Giving:
My best guess is that the world is on the cusp of changing radically — that we’ll look back at this period like we looked back on February 2020 before COVID-19 changed everything, except this time the transformation will be much more drastic and the world may never go back to how it was before.
Or Jake Mendel, who works with George:
I now think that we are more than 50% likely to have full blown superintelligence in the next 18 months, absent a slowdown. I need to call my family.
|
|
AI safety does have intellectual diversity: not everyone thinks like this, and those who do still disagree about timelines to AGI and the likelihood that things will go badly. But these views — that huge changes are coming soon and could be terrible — are common enough that they affect the way organisations work and plan.
Imagine running an AI safety org as someone who ‘takes AI seriously’:
- Hiring looks different if you think AI will have superhuman abilities in most areas by 2028.
- Fundraising looks different if you expect AI to produce dozens of billionaires.
- Everything looks different if a single model release might cut the org’s ‘time until Singularity’ estimate by six months.
If you can’t inhabit this state of mind, the worldview gap will make it harder to set the right priorities or coordinate with concerned colleagues.
You might also find the work environment strange or even grim. I’ve worked in offices where any discussion of the future after 2027 came with an invisible asterisk: “assuming we make it that far, and the world still makes sense.” I’ve hung out with new parents who fear that their children will inherit a world that has no place for them. I’ve seen people swap advice on finding a therapist who won’t be completely baffled by AGI-driven anxiety.
Again, you don’t need to share these concerns to do good work in AI safety. But you’ll fare better if you appreciate the perspective of colleagues who do. If you think that AGI transforming the world sounds like science fiction, you’ll struggle to thrive at some of our top-recommended organisations. (This career review shares more detail on why employers care about this.)
How to feel the AGI
Some of my favourite resources for understanding these worldviews:
- Levels of AGI-pilled presents a spectrum of how people think about AI, from “these are just fancy chatbots” to “this is as important as the Industrial Revolution” to “this is as big a deal as the emergence of life itself.” We don’t meet a lot of advisees at levels 0–1, but quite a few are sitting at 2, looking warily at 3. Meanwhile, most AI safety staffers are somewhere between 3 and 4, with many at 5.
- AI 2027 and AI 2040 present scenarios for how AI progress might play out. While no one agrees on every detail, many people think these scenarios offer plausible shapes for how things might go (from international treaties in the good timeline to engineered pandemics in the bad one).
- My most charismatic colleagues made a viral video about AI 2027, in case you want a quicker introduction!
- Preparing for the intelligence explosion takes a more academic, less story-driven approach to forecasting the future. It covers many different threats from transformative AI and ends with a section on how to prepare despite vast uncertainty.
- And finally, a different perspective: AI as normal technology argues that AI transformation isn’t likely, and gets a lot of respect even from people with the aforementioned worldviews. Read it for balance.
Beyond reading, you’ll want to:
- Use the best available AI models. We’ve seen tech journalists and world leaders form opinions on AI while using free models three generations behind. That’s like trying to judge the smartphone revolution from a BlackBerry. If you haven’t tried Opus 5.5 or Astra, subscribe for a month and give them your toughest problems. You may come out with a different sense of the future.
- Develop your own views. It’s easy to say you care about transformative AI and existential risk, but employers will want to see that you’ve weighed different perspectives for yourself and aren’t just copying something you’ve heard. (Including the views presented here!)
These are strange times. I think it’s important that we take the strangest possible futures seriously, and speak about them candidly. If you’re interested in working in AI safety, I hope this helps orient you to common views and vibes, so you aren’t taken by surprise as you explore further.
|
|
|
Featured release: new video podcast
|
|
|
Rob Wiblin discusses last month’s scandals at the three leading AI companies and why holding them accountable is now a real career path.
Watch the episode →
|
|
|
More new releases:
What we’ve been reading:
Opportunities to get involved:
- Stop the AI Race is organising a march to call for an international treaty to pause AI development until it is safe to proceed. You can join the march in the US (San Francisco, New York, Boston, Chicago, and Washington, DC), Canada (Toronto and Montreal), and France (Marseille and Paris).
- Taking place November 14.
- The Center on Long-Term Risk is accepting applications for its Fundamentals Program. This is an online reading programme on how transformative AI could lead to large-scale suffering and what can be done to reduce that risk.
- GovAI is accepting applications for its 2027 Summer Fellowships in Washington, DC and London (on a research or applied track). In this three-month paid programme, fellows will work on an AI governance research or applied project with support from GovAI’s team and network.
Have a great weekend,
Aaron
|
|
|
|
|
|