Rationality A-Z 1 Map and Territory
Ideas
A book by Eliezer Yudkowsky
We can't understand complex ideas: "When you explain things in an ancestral environment, you almost never have to explain your concepts. At most you have to explain one new concept, not two or more simultaneously."
It's hard to genuinely take other POVs: "In the ancestral environment, anyone who says something with no obvious support is a liar or an idiot. You're not likely to think, "Hey, maybe this person has well-supported background knowledge that no one in my band has even heard of"
Difference between an explanation and a password
Beliefs as cheering
"These are the signs of mysterious answers to mysterious questions: First, the explanation acts as a curiosity-stopper rather than an anticipation-controller. Second, the hypothesis has no moving parts—the model is not a specific complex mechanism, but a blankly solid substance or force. The mysterious substance or mysterious force may be said to be here or there, to cause this or that; but the reason why the mysterious force behaves thus is wrapped in a blank unity. Third, those who proffer the explanation cherish their ignorance; they speak proudly of how the phenomenon defeats ordinary science or is unlike merely mundane phenomena. Fourth, even after the answer is given, the phenomenon is still a mysteryand possesses the same quality of wonderful inexplicability that it had at the start." - Magic Answers - “Emergence” has become very popular, just as saying “magic” used to be very popular.
Rationality under Uncertainty still works
"Do not attempt long chains of reasoning or complicated plans."
Things are mysterious until they are not
Recognizing ourselves as the same as others. We are just as likely to make huge mistakes as other humans (kinda like !The Egg). We think oh it was just the Roman's who made that mistake, but that's us. We made that mistake and if we forget we will make it again. Remember how shocked we were when science revealed our beliefs weren't so mysterious, of course that could happen again.
"Don't imagine how you could have predicted the change, for that is amnesia. Remember that, in fact, you did not guess. Remember how, century after century, the world changed in ways you did not guess."
It doesn't matter that someone else already knows
"You probably believed that snow is white, don't you" "Um... yes?" "It doesn't bother you that Joseph Stalin believed that snow is white?" "Um... no?"
Selectively applying excuses instead of beliefs that are models of the world. Assuming the truth of those excuses only when it's convenient