Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

add pipeline parallelism to train llama #12

Closed
wanchaol opened this issue Jan 23, 2024 · 2 comments
Closed

add pipeline parallelism to train llama #12

wanchaol opened this issue Jan 23, 2024 · 2 comments
Assignees
@wconstab @kwen2501
Labels
enhancement New feature or request
Milestone
3D Parallel enabl...

Comments

@wanchaol
Copy link
Contributor

No description provided.

@lessw2020
Copy link
Contributor

I will start a sep branch for pippy integration and begin testing with this.

@tianyu-l tianyu-l added the enhancement New feature or request label May 3, 2024
@tianyu-l
Copy link
Contributor

3D parallel enabled in #344

jinsun-yoo pushed a commit to jinsun-yoo/torchtitan that referenced this issue Oct 30, 2024
@tianyu-l
test commit after rewriting history (pytorch#12)
3677bb2
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@wconstab wconstab

@kwen2501 kwen2501

Labels
enhancement New feature or request
Projects
None yet
Milestone
3D Parallel enablement
Development

No branches or pull requests
5 participants
@wconstab @kwen2501 @wanchaol @lessw2020 @tianyu-l