Artificial Intelligence Risk Mitigation ============================================ My main threat model is fast market agents in the hands of authoritarian or sociopathic/robber-baron hostile parties who attack economies and societies. We already have these in the form of high-speed trading corporations and disinformation campaigns by secret police, organize crime, and nationstates. But these could be carried out far faster and more effectively with another generation of machine learning / deep learning / whatver without requiring new tech or even that many resources. Really more of an augmentation thing than an AGI thing (though who knows how it could emerge). A lot of people seem to care/fear the "superintelligence" thing. I think this is a boogieman or red herring in most cases, but i'll also mention some things that could hedge against it, as much to see if people actually care about addressing this risk or are more interested in discussing how cool it would be or as a distraction for higher-expectation-value risks (like the above, or bio/chem/nuke weapons, or climate/ecological/resource collapse). ## Intensive Compute Intensive compute currently requires intensive energy consumption, and weird compute requires custom silicon. Both resources can be tracked. Silicon fabs are scarce; a neutral international body could review all output of high-end fabs looking for AI-specific devices. I think there are only like 5 regions/institutions in the world that do sub-20nm fabrication. Difficulties: - wouldn't it look a lot like bitcoin mining? (custom ASICs, huge power consumption) - AFAIK, in current tech effort is only around training, not actual deployment of neural net techniques. training can be async and distributed, executtion on small generic hardware? but could monitor "efforts"/research - intelligence agencies do a lot of sketchy monitoring using custom silicon and probably don't want to be monitored, even by a "neutral" body. Note that, unlike the nuclear weapons industry, intelligence agencies are probably commiting actual illegal/unaccountable acts, while weapons work was only secret to control spread of knowledge to the "enemy" and had civilian oversight Refs: Cory Doctorow (?) short story about an international monitoring service looking for waste heat from rogue/unlicensed "big data" operations, using satellite infrared cameras. ## Civic Institutional Resiliency If we consider AGI/superintelligence as a potentially threatening power, but only in abstract/informational ways to start, it seems obvious that general civic and infrastructural strength is a good hedge. Eg, core infrastructure air gapped from the net, defense-in-depth for networked devices, strong prevalent crypto (for things like government announcements, journalism, social media), robust voting systems, etc. Basically, look at the CIA/Putin handbook for disrupting other countries, and make sure we are more robust against those sort of "dirty" campaigns and manipulation. There's also technical resiliency: think it's pretty acknowledged that the current state of software and "security engineering" in particular are a general shitshow, almost everything has 0-days floating around, etc. Doing a bell-labs like effort to reset the norms, culture, and standards of the field (combined with clear guidelines and tools) could make software much more robust and secure (in my opinion). Bell labs was rare/expensive, but not *that* rare/expensive in the big picture (eg, compared to defense spending and gonzo secret projects). ## Slow Down Feedback Loops A commonly cited fear about superintelligence is that it could operate "really fast". There are a number of places in society that we could rate-limit and bring the tempo down to a human pace: - markets (trading) - changes to internet infrastructure, like BGP (largely in place already, I think) - almost all forms of beaurocracy or API could have sane rate limits Our legal/governance systems often have this baked in because those systems are already skeptical of "mobs" and disinformation. Checks and balances are a form of containment. A broader analysis of "power in the world" and having an early warning than any one entity (company, government, whatever) was gaining a controlling influence in any resource would be interesting as a general feedback safety thing. ## Culture I'm pretty confused about OpenAI, because it nominally is trying to de-risk AI, but it's basically just trying to advance the field (but be in the thick of it). It could instead visit labs around the world and issue reports, publish something like the "N minutes to midnight" (bullitin of atomic scientists), hold ethics debates and conferences, develop a code of ethics and get researchers to sign on, start student chapters at universities, lobby and consult with governments, draft regulations, call out "red flags" and tripwires, etc. Achieving broad cultural shift is hard, but way more leveraged than trying to get 100 people in a building to "solve the problem" or whatever. It's probably just the case the organization's goal is not what it's publicly state goal is (whether it knows that or not). Existence proofs of this strategy working are, I think, human cloning (pretty broad taboo; only a tiny fraction of people that could be are working on this AFAIK), chemical weapons, and to a large degree nuclear weapons (hard to recruit). ## Legibility Require automated systems controlling core infrastructure and markets to be human-meaningful: no black boxes. No neural nets with direct control over grid power pricing. This isn't directly out of fear that these systems would be "superintelligent" in their own, but that we wouldn't be able to debug and figure out if they had been manipulated, tampered with, or remotely reverse engineered in an info-crisis situation. Eg, a superintelligence is more likely to be able to understand and manipulate "black boxes" than we are. ## Comparison to Nuclear Regulation Monitoring and regulation of nuclear technologies seems to have been largely successful in tracking and observing (if not necessarily really containing proliferation or, most importantly, *reducing risk* of war as opposed to *preventing the growth of risk*). Things that maybe worked well then: - detection of test detonations in a variety of ways; almost impossible to hide? - advanced, specific machinery as a bottleneck - control/monitoring of physical materials - huge power and/or radiation required for refinement process - "delivery systems" monitored/controlled in parallel (and don't have difficulty) Refs: - Making of the Atomic Bomb, Dark Sun - Curve of binding energy (out of date but interesting as a snapshot in time) - Ellsberg's Doomsday Machine - Bertrand Russell book (TODO)