Skip to content

Issues: jiaweizzhao/GaLore

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clear current search query, filters, and sorts
37 Open 17 Closed
37 Open 17 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Resume function for optimizer
#35 opened Apr 3, 2024 by bokyeong1015
Any plan for the first stable release?
#36 opened Apr 8, 2024 by wsp317
Release of Trained Models
#38 opened Apr 9, 2024 by JLake310
can support llava model ?
#39 opened Apr 14, 2024 by awzhgw
ValueError: some parameters appear in more than one parameter group
#41 opened Apr 27, 2024 by jiaohuix
Questions about Figure 3 in the original paper
#42 opened May 1, 2024 by fy817
Galore finetuning #stopped
#51 opened Jun 7, 2024 by j-datta
Support for DDP with multi-gpus
#55 opened Jul 8, 2024 by seongjunyun
Questions about glue task report scores
#56 opened Jul 10, 2024 by MYT677
Results vs FP32
#59 opened Jul 30, 2024 by tsengalb99
loss figure data
#61 opened Sep 2, 2024 by BaohaoLiao
Question on Convergence and Grad Norm Behavior During Training with GaLore
#66 opened Nov 9, 2024 by chelouche9
Question on the estimated memory of GaLore
#67 opened Dec 14, 2024 by zqOuO
Please add Phi-2 Support
#19 opened Mar 13, 2024 by calebmor460
1
A few questions regarding the results and methodology.
#28 opened Mar 21, 2024 by roymiles
1
Support for Jamba (ai21labs/Jamba-v0.1)
#34 opened Apr 2, 2024 by creatorrr
1
How many GB memory is required to train the 7b model using DDP mode with galore?
#40 opened Apr 23, 2024 by zhangqijun
1
Galore unstable on Llama 7B beyond 20K steps
#43 opened May 2, 2024 by kyleliang919
1
torch_run.py lacking autocast and scaling for Automatic Mixed Precision
#45 opened May 9, 2024 by bhavnicksm
1
When I used galore on orpo, the learning rate was set to 8e-6, but the training rate was 0.01
#46 opened May 10, 2024 by Minami-su
1
Does galore save gradient memory?
#53 opened Jun 17, 2024 by jinqixiao
1
Figure 1 clarification on batch size and sequence length
#57 opened Jul 17, 2024 by psandovalsegura
1
Zero Loss: The algorithm failed to converge because the input matrix is ill-conditioned or has too many repeated singular values
#58 opened Jul 23, 2024 by akjindal53244
1
ValueError: can't optimize a non-leaf Tensor (param.is_leaf=False,param.retains_grad=False)
#60 opened Aug 21, 2024 by liveck
1
Training Time
#3 opened Mar 7, 2024 by thisisisheanesu
2
ProTip! What’s not been updated in a month: updated:<2024-12-31.