Skip to content

Issues: jiaweizzhao/GaLore

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clear current search query, filters, and sorts
38 Open 17 Closed
38 Open 17 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Any plan for the first stable release?
#36 opened Apr 8, 2024 by wsp317
Training Time
#3 opened Mar 7, 2024 by thisisisheanesu
2
Third-party benchmark
#6 opened Mar 7, 2024 by hiyouga
@jiaweizzhao
15
Double approximation of second moment in Adafactor
#8 opened Mar 7, 2024 by threewayhandshake
2
Hyperparameters for SFT?
#15 opened Mar 12, 2024 by peterjc123
4
Please add Phi-2 Support
#19 opened Mar 13, 2024 by calebmor460
1
GaLore in HuggingFace
#20 opened Mar 14, 2024 by IamExperimenting
12
How can i do continued pre-training using this?
#21 opened Mar 15, 2024 by Aloukik21
4
A few questions regarding the results and methodology.
#28 opened Mar 21, 2024 by roymiles
1
Reproducing Perplexity evaluation
#30 opened Mar 22, 2024 by NitzanHod
2
Dataset loading issue, integration with Colossal-AI
#33 opened Mar 29, 2024 by Edenzzzz
3
Support for Jamba (ai21labs/Jamba-v0.1)
#34 opened Apr 2, 2024 by creatorrr
1
support sft?
#1 opened Mar 7, 2024 by NickyDark1
3
Resume function for optimizer
#35 opened Apr 3, 2024 by bokyeong1015
Release of Trained Models
#38 opened Apr 9, 2024 by JLake310
can support llava model ?
#39 opened Apr 14, 2024 by awzhgw
How many GB memory is required to train the 7b model using DDP mode with galore?
#40 opened Apr 23, 2024 by zhangqijun
1
ValueError: some parameters appear in more than one parameter group
#41 opened Apr 27, 2024 by jiaohuix
Questions about Figure 3 in the original paper
#42 opened May 1, 2024 by fy817
Galore unstable on Llama 7B beyond 20K steps
#43 opened May 2, 2024 by kyleliang919
1
Questions about reproducing the result of "Benchmark 2: Fine-Tuning RoBERTa on GLUE tasks"
#44 opened May 4, 2024 by JamesSand
2
torch_run.py lacking autocast and scaling for Automatic Mixed Precision
#45 opened May 9, 2024 by bhavnicksm
1
When I used galore on orpo, the learning rate was set to 8e-6, but the training rate was 0.01
#46 opened May 10, 2024 by Minami-su
1
IndexError: tuple index out of range
#47 opened May 13, 2024 by zyushun
11
Question on the estimated memory of GaLore
#67 opened Dec 14, 2024 by zqOuO
ProTip! What’s not been updated in a month: updated:<2024-12-19.