The readiness for spontaneous cooperation together with the assumptions that others share this cooperativity has been identified as a fundamental feature that distinguishes humans from other animals, including the great apes. At the same time, cooperativity presents an evolutionary puzzle because non-cooperators do better in a group of cooperators. We develop here an analysis of the process leading to cooperation in terms of rationality concepts, game theory and epistemic logic. We are, however, not attempting to reconstruct the actual evolutionary process. We rather want to provide the logical structure underlying cooperation in order to understand why cooperation is possible and what kind of reasoning and beliefs would lead to cooperative decision-making. Game theory depends on an underlying common belief in non-cooperative rationality of the players, and cooperativity similarly can utilize a common belief in cooperative rationality as its basis. We suggest a weaker concept of rational decision-making in games that encompasses both types of decision-making. We build this up in stages, starting from simple optimization, then using anticipation of the reaction of others, to finally arrive at reflexive and cooperative reasoning. While each stage is more difficult than the preceding, importantly, we also identify a reduction of complexity achieved by the consistent application of higher stage reasoning.