[Linkpost] “Concrete projects to prepare for superintelligence” by Forethought, William_MacAskill, finm
3mo agoen
Source
This is a link post. Introduction There are lots of good, neglected, and pretty concrete projects people could set up to make the transition to superintelligence go better. This document describes some that readers might not have thought much about before. They are ordered roughly by how excited we are about them.[1] Of these, Forethought is actively working on AI character evaluation and space governance, and we are very interested in automating macrostrategy. Summary AI character evaluation. Start an independent org to evaluate and stress-test AI character traits (epistemic integrity, prosociality, appropriate refusals), hold developers accountable against their own model specs / constitutions, and suggest and incentivise improvements to the specs. Automated macrostrategy. Create evaluations and benchmarks, collect human-generated training data, and build scaffolds to improve AI competence at big-picture strategic and philosophical reasoning. AI security assessment. Start an independent org that evaluates AI models for sabotage and backdoors, and makes recommendations about AI constitutions. Enabling deals. Start an independent organisation to broker deals with potentially misaligned AI models in order to incentivise early schemers to disclose misalignment and cooperate with alignment efforts. AI for improving collective epistemics. E.g. build an AI chief of staff that helps users act in [...] --- Outline: (00:14) Introduction (00:43) Summary (02:31) AI character evaluation (04:40) Automated macrostrategy (08:09) AI security evaluations (09:11) Enabling deals with AIs (11:14) Tools for collective epistemics (13:11) Epistemic tools for strategic awareness (14:26) Tools for coordination (16:16) Space governance institute (19:39) Coalition of concerned ML scientists --- First published: March 27th, 2026 Source: Linkpost URL: --- Narrated by TYPE III AUDIO .
You might also wanna read
Recursive self-improvement: Could AI superintelligence arrive within years?
This article discusses the concept of recursive self-improvement in AI, where AI models could build increasingly capable successors without
Bruce Schneier: AI and the scaling of betrayal
schneier.com·5mo ago
Recursive self-improvement: Could AI build its own superintelligent successor within years?
This article discusses the concept of recursive self-improvement in AI, where an AI model builds a more advanced successor without human int
Recursive self-improvement: Could AI build its own superintelligent successor within years?
This article discusses the concept of recursive self-improvement in AI, where an AI model builds a more advanced successor without human int
New Nick Bostrom Paper: Optimal Timing for Superintelligence [pdf]
nickbostrom.com·4mo ago
Thinking Fast, Slow, and Artificial: How AI Is Reshaping Human Reasoning
papers.ssrn.com·3mo ago
Analyzing a Critique of the AI 2027 Timeline Forecasts
thezvi.substack.com·1y ago

Comments
Sign in to join the conversation.
No comments yet. Be the first.