I.

In 1980, game theorist Robert Axelrod ran a famous Iterated Prisoner’s Dilemma Tournament.

He asked other game theorists to send in their best strategies in the form of “bots”, short pieces of code that took an opponent’s actions as input and returned one of the classic Prisoner’s Dilemma outputs of COOPERATE or DEFECT. For example, you might have a bot that COOPERATES a random 80% of the time, but DEFECTS against another bot that plays DEFECT more than 20% of the time, except on the last round, where it always DEFECTS, or if its opponent plays DEFECT in response to COOPERATE.

In the “tournament”, each bot “encountered” other bots at random for a hundred rounds of Prisoners’ Dilemma; after all the bots had finished their matches, the strategy with the highest total utility won.

To everyone’s surprise, the winner was a super-simple strategy called TIT-FOR-TAT:

1. Always COOPERATE on the first move.

2. Then do whatever your opponent did last round.

This was so boring that Axelrod sponsored a second tournament specifically for strategies that could displace TIT-FOR-TAT. When the dust cleared, TIT-FOR-TAT still won - although some strategies could beat it in head-to-head matches, they did worst against each other, and when all the points were added up TIT-FOR-TAT remained on top.

In certain situations, this strategy is dominated by a slight variant, TIT-FOR-TAT-WITH-FORGIVENESS. That is, in situations where a bot can “make mistakes” (eg “my finger slipped”), two copies of TIT-FOR-TAT can get stuck in an eternal DEFECT-DEFECT equilibrium against each other; the forgiveness-enabled version will try cooperating again after a while to see if its opponent follows. Otherwise, it’s still state-of-the-art.

The tournament became famous because - well, you can see how you can sort of round it off to morality. In a wide world of people trying every sort of con, the winning strategy is to be nice to people who help you out and punish people who hurt you. But in some situations, it’s also worth forgiving someone who harmed you once to see if they’ve become a better person. I find the occasional claims to have successfully grounded morality in self-interest to be facile, but you can at least see where they’re coming from here. And pragmatically, this is good, common-sense advice.

For example, compare it to one of the losers in Axelrod’s tournament. COOPERATE-BOT always cooperates. A world full of COOPERATE-BOTS would be near-utopian. But add a single instance of its evil twin, DEFECT-BOT, and it folds immediately. A smart human player, too, will easily defeat COOPERATE-BOT: the human will start by testing its boundaries, find that it has none, and play DEFECT thereafter (whereas a human playing against TIT-FOR-TAT would soon learn not to mess with it). Again, all of this seems natural and common-sensical. Infinitely-trusting people, who will always be nice to everyone no matter what, are easily exploited by the first sociopath to come around. You don’t want to be a sociopath yourself, but prudence dictates being less-than-infinitely nice, and reserving your good nature for people who deserve it.

Reality is more complicated than a game theory tournament. In Iterated Prisoners’ Dilemma, everyone can either benefit you or harm you an equal amount. In the real world, we have edge cases like poor people, who haven’t done anything evil but may not be able to reserve your generosity. Does TIT-FOR-TAT help the poor? Stand up for the downtrodden? Care for the sick? Domain error; the question never comes up.

Still, even if you can’t solve every moral problem, it’s at least suggestive that, in those domains where the question comes up, you should be TIT-FOR-TAT and not COOPERATE-BOT.

This is why I’m so fascinated by the early Christians. They played the doomed COOPERATE-BOT strategy and took over the world.

II.

Matthew 5:

You have heard that it was said, ‘Love your neighbor and hate your enemy.’ But I tell you, love your enemies and pray for those who persecute you . . . If you love those who love you, what reward will you get? Are not even the tax collectors doing that? And if you greet only your own people, what are you doing more than others? Do not even pagans do that?

Talk is cheap, but The Rise Of Christianity suggests the early Christians pulled it off. For example, even though pagan institutions would not help indigent Christians, Christians tried to give charity to Christian and pagan alike, even going so far as to help nurse pagans during the plague (when nursing a victim conferred a high risk of contagion and death). Even Emperor Julian, an enemy of Christianity, admitted it lived up to its own standards:

When the poor happened to be neglected and overlooked by the priests, the impious Galileans observed this and devoted themselves to benevolence . . . [they] support not only their poor, but ours as well, [when] everyone can see that our people lack aid from us.”

In 1 Corinthians 6, Paul is asked whether it is acceptable for one Christian to pursue a lawsuit against another Christian in a pagan court. He answers:

The very fact that you have lawsuits among you means you have been completely defeated already. Why not rather be wronged? Why not rather be cheated?

We get a similar picture from the stories of the martyrs. Many of them prayed for the Romans while the Romans were in the process of torturing and killing them; Polycarp even cooked them a meal.

If the Christians had merely been TIT-FOR-TAT, it would be easy to tell a story of their victory. The Roman Empire was corrupt and decadent to the core. People were looking for a community they could trust. Christianity offered access to a better class of friends who wouldn’t immediately rob or betray you when your guard was down. By providing a superior alternative to the low-trust pagan world, it was irresistible on a purely rational economic basis.

But this story sounds more worthy of the mystery cults. Mystery cults are a great structure for mutual aid; we see this today in groups like the Freemasons (cf. Backscratcher Clubs). Everybody knows who’s on the inside (and needs to be mutually aided) and who’s on the outside (and can be ignored). The initiatory structure holds off freeloaders and makes sure the people on the inside are of approximately equal rank (so that you get as many benefits as you give) and can be held accountable if they don’t contribute.

Since Christianity did better than the mystery cults, there must have been some reason that COOPERATE-BOT beat TIT-FOR-TAT in the particular environment of Roman religion, defying all normal game theoretic logic.

III.

Is this a consistent feature of COOPERATE-BOT strategies, or was it just luck?

This is hard to say, because in all normal cases it’s impossible to follow a COOPERATE-BOT strategy at scale and for any period of time.

Consider the Quakers, who gave it a better try than most. They were persecuted by the British and fled to America (is this kosher? it sort of seems like resisting evil). There they founded the colony of Pennsylvania, intended to be a utopia of pacifism and benevolence. They were very serious about this; history records many Quakers who were arrested or even killed rather than compromise their principles, and the British Crown seized Pennsylvania from the Quakers a few times because they wouldn’t make extremely cheap gestures like pay taxes or swear oaths.

But in the end, the Crown frog-boiled the Quakers into compliance. They promised to return self-government if the Quakers would budge an inch - in one compromise, if they agreed to pay taxes that could go to non-combat functions of the military. The Quakers eventually agreed, and the British ratcheted up their demands the next time. Finally, in 1755, some Indians launched a major assault on Pennsylvania, and all the Quakers voluntarily resigned from government to let the non-Quaker Pennsylvanians (who by this time outnumbered them) conduct the war without restraint.

The Quakers performed better than most COOPERATE-BOTs. They stuck to their principles most of the time, and in the end their religion survived. But look deeper, and you see a gradual process of surrender to reality. First was the flight to America, an implicit admission that living was better than being martyred for the faith. Then came the various compromises; an implicit admission that getting to keep self-government while being 99% pure was better than being subjects while 100% pure. Finally, they gave up Pennsylvania itself rather than be wiped out, again choosing the practical option over martyrdom. My point isn’t to knock the Quakers, who may come in a close 2nd in “historical groups that stuck to their cooperative principles despite all odds” and were certainly more ethical than I am. My point is that even very committed groups of religious fanatics fail the non-violent COOPERATE-BOT strategy eventually.

Or maybe the ones who didn’t fail were wiped out? I hear good things about the Cathars, but we can’t know for sure because they were very thoroughly killed off - unrepentant to the last.

Are there any other groups who deserve mention in this section besides early Christians, Quakers, and Cathars? I think some German and Russian sects have tried similar strategies, though they mostly failed and I don’t know much about them.

Not exactly the same, but maybe rhyming: what about modern liberalism? To the monarchs and dictators of the past, free speech might seem kind of like COOPERATE-BOT in a limited domain: the idea that elites shouldn’t make any forceful/legal effort to protect their ideological and spiritual position must sound almost as crazy as them not making any forceful/legal effort to protect themselves if attacked, or to prevent themselves from getting cheated. It is, in some sense, a unilateral surrender in the war of ideas; fascists and communists will do their best to crush liberalism, but liberals cannot ban discussion of fascism or communism. The fact that this, too, has worked, makes me think early Christianity wasn’t just a one-off, but suggests some larger point.

IV.

Still, I don’t really know what it is. Here are some weak theories:

1. Advertisement: Being kind to outsiders is good PR and encourages those outsiders to join you. This effect is stronger than the corresponding disincentive (that they won’t get much better treatment than they’re getting already, and they will have to be nice to other outsiders in their turn).

2. Selecting a moral elite: The only people willing to put up with the COOPERATE-BOT strategy are hyperscrupulous saints. A movement that starts out with hyperscrupulous saints is naturally high-trust and admirable. The benefits of filtering for these people outweigh the obvious cost (namely, that most people won’t join your movement, or will burn out).

3. Overwhelm downward adjustment: If you assume all movements lie and downgrade their claims 90% out of cynicism, then a movement which merely portrays itself as helping members won’t even help members, but a movement which portrays itself as radiating universal love to all mankind might at least help its members.

4. Eliminate transaction costs from means-testing: In a typical backscratchers club, most of the social mores evolve as ways to eliminate free-riders. If you explicitly accept free-riders, you can cut a lot of red tape and present a much more accepting environment.

5. Maybe people are actually good: Maybe the liberals are right about everything, and most human evil comes from misunderstanding + a sense of being excluded. If you’re so accepting of everybody that misunderstandings don’t matter, and you don’t exclude people, then people mostly don’t try to take advantage of you or give you trouble.

6. Greater psychological appeal: Maybe “be infinitely nice all the time” is more attractive and psychologically stable than “be nice in X, Y, Z circumstances”, and the sheer endorphin rush people get from letting their moral impulses run wild is so addictive that it outweighs the obvious cost.

7. Greater heroic appeal: Similar to the above. Historians of war have remarked that you don’t just inspire soldiers by saying “don’t worry, this campaign will be easy”. You can sometimes inspire them by saying that this will be the most difficult thing they’ve ever done, and it’s heroic for them to even consider such an enterprise. In the same way, maybe “give 1% more to the worthy poor” is boring, but “devote your entire life to loving everybody including your enemies” is shocking and heroic enough to excite a certain type of person.

8. Something something limits of prediction: It’s a truism that naive consequentialism doesn’t actually have the best consequences; for example, it may seem like a good idea to steal money and then give it to the poor, but (as SBF et al discovered) this ends up being good neither for you nor for the poor, since you eventually make a miscalculation, get discovered, turn people against you, turn people against your cause, etc. Even if you try really hard to make a clever plan rather than a dumb plan, you’ll eventually make a mistake and your clever plan will crash and burn. Therefore, even if you’re a deep-seated consequentialist, you should avoid acting as a consequentialist and instead follow normal-person morality. But maybe there’s a second, deeper layer to this insight. Maybe even following normal-person morality is trying too hard to be clever and galaxy-brained in a way that never works. Maybe, it too, collapses into counterproductivity after you make your first bad prediction. Maybe you should actually be following COOPERATE-BOT morality instead.

9. Epiphenomenal: COOPERATE-BOT isn’t really a good strategy, but is an unavoidable side effect of something else valuable. For example, maybe you couldn’t have Christians who loved God so much without having them be extremely loving and charitable people. The most dramatic version of this hypothesis is that God is real, and loving thy enemy is an epiphenomenon of following the actual Divine Law.

Do I really believe any of these?

I guess that question cashes out to “if you were involved in a movement, would you recommend COOPERATE-BOT as a strategy today?” The movements I’m actually involved in (rationalists, effective altruists) occasionally have slightly related debates. One of them involves PR: a pragmatist faction wants to stay away from hit-piece-writers, network with friendly journalists to ensure positive coverage, keep our best side forward, and de-emphasize (not deny or lie about) embarrassing bad sides. A COOPERATE-BOT faction thinks that’s what the Pharisees and tax collectors are doing, but that we’re trying to be more epistemically cooperative than everyone else and it’s our responsibility to just dump the exact contents of our brains out to anybody who asks us any question, without regard for the consequences.

There’s a parallel debate in charity funding. A pragmatist faction wants to make sure everything we fund is PR-friendly and won’t make everybody hate us or be incredibly embarrassing if it fails; a COOPERATE-BOT faction thinks we have a moral duty to fund the exact object-level highest-utility projects even if everyone will hate us for it and we’ll never get another penny of funding every again. I wrote up an allegorical history of this conflict here. I lean towards the pragmatist side of most of these conflicts, if only because I’ve seen enough PR disasters to know that nobody gives you any slack for having stumbled into them only because of your exceptional moral purity.

But even without endorsing the full strategy, there’s a vibe there that I really like. Whenever I discuss moral issues, people in the comments section here will do the whole post-Christian Nietzschean thing: “If you admit moral obligations to people who can’t pay you back, aren’t you just cucked? Aren’t you unilaterally surrendering in this memetic war we’re in, destined to be replaced by civilizations/ideologies with more continence in limiting their altruism to useful people bound in bilateral contracts? Who the heck cares if some foreigner or animal is suffering? Isn’t that just pathological, a proof that you don’t have the steely will that it takes to survive?”

I admit these people’s position makes rational sense. But on the deepest level, I don’t believe it. There was an old 2010s meme video about all the characters in all the comics and TV shows fighting, and in the end the one who came out on top was “Mr. Rogers, in a bloodstained sweater”. Human history is the cosmic version of that meme. After all the Vikings and steppe nomads and Spartans have had their way with each other, the leading ideology of the 21st century thus far appears to be a hyper-Christian bleeding-heart liberalism: COOPERATE-BOT in a bloodstained sweater. I don’t know why this keeps happening, but I wouldn’t count it out.