Commits

Breadcrumbs

History for

onmain

User selector

All users

Datepicker

All time

Commit History

Commits on Dec 18, 2024

• Upgrade Mistral MoE and Dense model training (alibaba#400)

  Show description for 011e96d
  

  jerryli1981

  and

  同润

  authored
  011e96d

  Copy full SHA for 011e96d

Commits on Dec 5, 2024

• LLama2 70B upgrade and fix nan issue (alibaba#391)

  Show description for 9b88cc4
  

  jerryli1981

  and

  同润

  authored
  9b88cc4

  Copy full SHA for 9b88cc4

Commits on Nov 27, 2024

• Add preview implementation of Qwen2-VL in PAI-Megatron-Patch (alibaba#389)

  lostkevin

  authored
  11cf249

  Copy full SHA for 11cf249

Commits on Nov 26, 2024

• bugfix qwen2 grouped gemm error (alibaba#388)

  Show description for abe462c
  

  jamindy

  and

  james

  authored
  abe462c

  Copy full SHA for abe462c

Commits on Nov 21, 2024

• Add LLaVa mcore implementation (alibaba#384)

  Show description for 51d49f0
  

  jerryli1981

  and

  同润

  authored
  51d49f0

  Copy full SHA for 51d49f0

Commits on Nov 18, 2024

• Fix hang issue due to mismatched attention mask (alibaba#379)

  Show description for 3979dbb
  lostkevin

  authored
  3979dbb

  Copy full SHA for 3979dbb

Commits on Nov 14, 2024

• Fix broadcast CPU tensor issue when TP>1 PP=1 in jsonl dataset SFT (alibaba#377)

  Show description for 32342a8
  lostkevin

  authored
  32342a8

  Copy full SHA for 32342a8

Commits on Oct 30, 2024

• Add llm auto configurator and apply per seq sft loss for qwen2/2.5 models (alibaba#371)

  Show description for bf582d8
  

  jerryli1981

  and

  同润

  authored
  bf582d8

  Copy full SHA for bf582d8

Commits on Oct 24, 2024

• Fix DeepSeekV2Tokenizer bug that padding_side is not set to right (alibaba#370)

  Show description for 8003e0c
  

  jerryli1981

  and

  同润

  authored
  8003e0c

  Copy full SHA for 8003e0c

Commits on Sep 26, 2024

• Upgrade deepseek-v2-moe models to support MLA via transformer engine and pipeline ckpts conversion (alibaba#355)

  Show description for bcb69cb
  

  jerryli1981

  and

  同润

  authored
  bcb69cb

  Copy full SHA for bcb69cb

Commits on Sep 22, 2024

• update sequence packing and add qwen2.5 (alibaba#349)

  Show description for daf5c82
  lostkevin

  authored
  daf5c82

  Copy full SHA for daf5c82

Commits on Sep 12, 2024

• Implement Sequence Packing in SFT for Qwen2 and LlaMA-3.1 models (alibaba#344)

  Show description for 7e9deaa
  lostkevin

  authored
  7e9deaa

  Copy full SHA for 7e9deaa

Commits on Sep 11, 2024

• feature: support for qwen2Toknizer data preprocess and pretraining (alibaba#337)

  Show description for b3a7df8
  village-way

  authored
  b3a7df8

  Copy full SHA for b3a7df8

Commits on Sep 10, 2024

• fix/rm eos_token_id mask (alibaba#335)

  wakafengfan

  authored
  dd20470

  Copy full SHA for dd20470

Commits on Aug 26, 2024

• Upgrade qwen2 dense and moe models to support Flash-Attention 3, Offloading, Comm-Overlapping features (alibaba#328)

  Show description for 230f1f6
  

  jerryli1981

  and

  同润

  authored
  230f1f6

  Copy full SHA for 230f1f6

Commits on Aug 23, 2024

• Implement LLaMA3.1 in Pai-Megatron-Patch (alibaba#326)

  Show description for 3ea909d
  lostkevin

  authored
  3ea909d

  Copy full SHA for 3ea909d

Commits on Jul 25, 2024

• Add auto optimizer offloading and update ReadMe (alibaba#298)

  lostkevin

  authored
  939ab1d

  Copy full SHA for 939ab1d

Commits on Jul 24, 2024

• enable finetune with idxmap datasets (alibaba#297)

  Show description for 21942e3
  

  jerryli1981

  and

  同润

  authored
  21942e3

  Copy full SHA for 21942e3

Commits on Jul 18, 2024

• Update CPU Offload Optimizer (alibaba#290)

  Show description for 75fb1d4
  

  jerryli1981

  and

  同润

  authored
  75fb1d4

  Copy full SHA for 75fb1d4

Commits on Jul 11, 2024

• Add MPI Support for tp-comm-overlap and Cpu-Offload for Mcore Distributed Optimizer in LLama3 (alibaba#283)

  Show description for fc1a002
  

  jerryli1981

  and

  同润

  authored
  fc1a002

  Copy full SHA for fc1a002

Commits on Jun 23, 2024

• Fix Qwen2 MoE Loss Convergence Issue (alibaba#275)

  Show description for 0ac48a0
  

  jerryli1981

  and

  同润

  authored
  0ac48a0

  Copy full SHA for 0ac48a0

Commits on Jun 21, 2024

• Add Floating Point Control for Qwen2 Model Convertor (alibaba#274)

  Show description for 4c94f2e
  

  jerryli1981

  and

  同润

  authored
  4c94f2e

  Copy full SHA for 4c94f2e

Commits on Jun 20, 2024

• Enhance Qwen2 Model Finetune Script and Make Qwen2Tokenzier callable (alibaba#273)

  Show description for b9a9e7b
  

  jerryli1981

  and

  同润

  authored
  b9a9e7b

  Copy full SHA for b9a9e7b

Commits on Jun 19, 2024

• Add Qwen2 MoE Model Mcore Implementation (alibaba#267)

  Show description for 2525333
  

  jerryli1981

  and

  同润

  authored
  2525333

  Copy full SHA for 2525333

Commits on Jun 18, 2024

• Qwen1&Qwen1.5&Qwen2 tokenizer fix (alibaba#266)

  Show description for e3fc455
  Carrie-Yi

  authored
  e3fc455

  Copy full SHA for e3fc455

Commits on Jun 13, 2024

• Add Qwen2 Dense Models Forward Loss Evalution Checking (alibaba#259)

  Show description for 0f53875
  

  jerryli1981

  and

  同润

  authored
  0f53875

  Copy full SHA for 0f53875

Commits on Jun 12, 2024

• Add Qwen2 Dense Models Mcore Implementation (alibaba#258)

  Show description for ea917f9
  

  jerryli1981

  and

  同润

  authored
  ea917f9

  Copy full SHA for ea917f9

Commits on Jun 11, 2024

• Fix DeepSeep-V2 Tensor Parallel MLA Loss Issue (alibaba#254)

  Show description for 7097866
  

  jerryli1981

  and

  同润

  authored
  7097866

  Copy full SHA for 7097866

Commits on Jun 10, 2024

• DeepSeek-V2-MoE transformer config adapt to mcore0.6.0 (alibaba#253)

  Show description for 942c562
  

  one-game

  and

  one_game

  authored
  942c562

  Copy full SHA for 942c562

Commits on May 31, 2024

• Update DeepSeek-V2-MoE ReadMe and tensor parallel mla support (alibaba#245)

  Show description for 8be40a3
  

  jerryli1981

  and

  同润

  authored
  8be40a3

  Copy full SHA for 8be40a3

Commits on May 29, 2024

• Qwen-1.5-MoE router strategy alignment with huggingface implementation (alibaba#241)

  Show description for 3de1394
  

  jerryli1981

  and

  同润

  authored
  3de1394

  Copy full SHA for 3de1394

Commits on May 27, 2024

• Add DeepSeek-V2-MoE Mcore implementation (alibaba#240)

  Show description for a002437
  

  jerryli1981

  and

  同润

  authored
  a002437

  Copy full SHA for a002437

Commits on May 23, 2024

• Decoupled llama3 and qwen1.5 mcore implementation (alibaba#238)

  Show description for 915a8cd
  

  jerryli1981

  and

  同润

  authored
  915a8cd

  Copy full SHA for 915a8cd

Commits on May 13, 2024

• Add fine-grained moe support for qwen1.5 (alibaba#227)

  Show description for 1577d2b
  

  jerryli1981

  and

  同润

  authored
  1577d2b

  Copy full SHA for 1577d2b

Commits on Apr 21, 2024

• Update Quick Starts for LLama3 (alibaba#193)

  Show description for eed0fe6
  

  jerryli1981

  and

  同润

  authored
  eed0fe6

  Copy full SHA for eed0fe6

Pagination

Previous

Next