The burgeoning interest in developing Large Language Models (LLMs) with up to trillion
parameters has been met with concerns regarding resource efficiency and practical
expense, particularly given the immense cost of experimentation. This scenario underscores
the importance of exploring the potential of Small Language Models (SLMs) as a resource-
efficient alternative. In this context, we introduce MiniCPM, specifically the 1.2 B and 2.4 B
non-embedding parameter variants, not only excel in their respective categories but also …