A Native-PyTorch Library for LLM Fine-tuning

Modeling, training, eval, and inference code for OLMo

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

Repo to hold code and track issues for the collection of permissively licensed data

Repo for the paper "Large Language Models Struggle to Learn Long-Tail Knowledge"

An implementation of model parallel autoregressive transformers on GPUs, based on the DeepSpeed library.

Task-based datasets, preprocessing, and evaluation for sequence models.

Reproduce results and replicate training fo T0 (Multitask Prompted Training Enables Zero-Shot Task Generalization)

A framework for few-shot evaluation of autoregressive language models.

A fork of a fork of a fork of T5x Repo for my own experiments

Toolkit for creating, sharing and using natural language prompts.

Ongoing research training transformer language models at scale, including: BERT & GPT-2

Beyond the Imitation Game collaborative benchmark for measuring and extrapolating the capabilities of language models