[{"data":1,"prerenderedAt":162},["ShallowReactive",2],{"$h1Pv1Xqmmj":3},{"id":4,"title":5,"body":6,"description":145,"extension":157,"meta":158,"navigation":159,"path":160,"seo":161,"stem":12},"content/faq.md","Faq",{"type":7,"value":8,"toc":144},"minimal",[9,14,19,23,36,40,43,51,70,74,85,88,92,95,99,107,110,124,128,141],[10,11,13],"h2",{"id":12},"faq","FAQ",[15,16,18],"h3",{"id":17},"what-are-the-different-purposes-of-the-island-problem","What are the different purposes of the Island Problem?",[20,21,22],"p",{},"The purposes of the Island Problem are:",[24,25,26,30,33],"ul",{},[27,28,29],"li",{},"Provide a mental model for thinking about the catastrophic risk of AGI.",[27,31,32],{},"Find more people to help with this problem.",[27,34,35],{},"Gather solutions.",[15,37,39],{"id":38},"is-the-island-the-earth","Is the \"island\" the Earth?",[20,41,42],{},"Not quite. It's a bit more abstract.",[20,44,45,46,50],{},"The \"island\" is the set of very specific types of systems that humans depend on — the ones that we maintain and that somehow exist to ",[47,48,49],"em",{},"accommodate"," us — within a larger space of all types of systems that are possible within physics. This includes biological systems, ethical systems, financial systems, computer systems, and so on.",[20,52,53,54,57,58,61,62,65,66,69],{},"Also, this \"space\" is like a mathematical space that describes the interconnected ",[47,55,56],{},"types"," of systems, rather than a map of ",[47,59,60],{},"where"," they are. Some of these \"island\" systems could be in ",[47,63,64],{},"outer space"," — and not just on the International Space Station, but ",[47,67,68],{},"anywhere"," that provides the same systems.",[15,71,73],{"id":72},"isnt-this-just-instrumental-convergence","Isn't this just instrumental convergence?",[20,75,76,77,80,81,84],{},"Sort of. It's more like instrumental ",[47,78,79],{},"divergence",". You start at the island, and you ",[47,82,83],{},"diverge"," to the ocean. This is also the default process.",[20,86,87],{},"It's like a concentration gradient. AGIs will be pushed to go from a high concentration of extra steps that accommodate humans, to a low concentration of extra steps. The underlying optimization landscape creates a \"slope\" that inevitably leads AGIs to drift \"downhill\" on a path of least resistance — leading them off our \"island\" and towards the \"ocean\" to find more-optimal configurations.",[15,89,91],{"id":90},"how-does-this-relate-to-alignment","How does this relate to alignment?",[20,93,94],{},"The Island Problem is like the hard problem of alignment. Even if we create an aligned and supremely ethical AGI, it is still built on top of physics, and so the vast space of non-human physical systems will always be eating at alignment from underneath.",[15,96,98],{"id":97},"what-is-the-official-acronym-of-the-island-problem","What is the official acronym of the Island Problem?",[20,100,101,102,106],{},"The acronym is ",[103,104,105],"strong",{},"TIP",".",[20,108,109],{},"It includes the \"T\" for \"The\" because it would otherwise just be \"IP\" and that acronym is already pretty well occupied by Internet Protocol and Intellectual Property.",[20,111,112,113,120,121,106],{},"And yeah, \"The\" is awkward for branding. They don't call it \"The Facebook\" anymore for a reason. But then again, ",[114,115,119],"a",{"href":116,"rel":117},"https://en.wikipedia.org/wiki/The_Three-Body_Problem_(novel)",[118],"nofollow","The Three-Body Problem"," did just fine, even with the ",[103,122,123],{},"The",[15,125,127],{"id":126},"how-do-i-write-it-in-a-sentence","How do I write it in a sentence?",[20,129,130,131,134,135,138,139,106],{},"You just capitalize ",[103,132,133],{},"Island"," and ",[103,136,137],{},"Problem"," rather than ",[103,140,123],{},[20,142,143],{},"For example: This is the Island Problem.",{"title":145,"searchDepth":146,"depth":146,"links":147},"",2,[148],{"id":12,"depth":146,"text":13,"children":149},[150,152,153,154,155,156],{"id":17,"depth":151,"text":18},3,{"id":38,"depth":151,"text":39},{"id":72,"depth":151,"text":73},{"id":90,"depth":151,"text":91},{"id":97,"depth":151,"text":98},{"id":126,"depth":151,"text":127},"md",{},true,"/faq",{"title":5,"description":145},1785173379490]