Skip to content

Commit

Permalink
Update README.md
Browse files Browse the repository at this point in the history
  • Loading branch information
Manchery committed Jun 7, 2023
1 parent 5431f7f commit c846eb8
Showing 1 changed file with 4 additions and 1 deletion.
5 changes: 4 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ Please let me know if you find any mistakes or omissions! Your contribution is w
- [Others](#others)
- [Optimization](#optimization)
- [Loss & Gradient Strategy](#loss--gradient-strategy)
- [Task Interference](#task-interference)
- [Task Sampling](#task-sampling)
- [Adversarial Training](#adversarial-training)
- [Pareto](#pareto)
Expand Down Expand Up @@ -244,13 +245,15 @@ Please let me know if you find any mistakes or omissions! Your contribution is w
### Loss & Gradient Strategy

- **[MoCo]** Fernando, H. D., Shen, H., Liu, M., Chaudhury, S., Murugesan, K., & Chen, T. [Mitigating Gradient Bias in Multi-objective Learning: A Provably Convergent Approach](https://openreview.net/forum?id=dLAYGdKTi2). ICLR, 2023.
- **[FAMO]** Liu, B., Feng, Y., Stone, P., & Liu, Q. [FAMO: Fast Adaptive Multitask Optimization](http://arxiv.org/abs/2306.03792). ArXiv, 2023.
-**[ForkMerge]** Jiang, J., Chen, B., Pan, J., Wang, X., Dapeng, L., Jiang, J., & Long, M. [ForkMerge: Overcoming Negative Transfer in Multi-Task Learning](http://arxiv.org/abs/2301.12618). ArXiv, 2023.
- **[AuxiNash]** Shamsian, A., Navon, A., Glazer, N., Kawaguchi, K., Chechik, G., & Fetaya, E. [Auxiliary Learning as an Asymmetric Bargaining Game](https://arxiv.org/abs/2301.13501). ArXiv, 2023.
- ✨ Xin, Derrick, Behrooz Ghorbani, Justin Gilmer, Ankush Garg, and Orhan Firat. **[Do Current Multi-Task Optimization Methods in Deep Learning Even Help?](https://openreview.net/forum?id=A2Ya5aLtyuG)** NeurIPS, 2022.
- **[Unitary Scalarization]** Kurin, V., De Palma, A., Kostrikov, I., Whiteson, S., & Kumar, M. P. [In Defense of the Unitary Scalarization for Deep Multi-Task Learning](http://arxiv.org/abs/2201.04122). NeurIPS, 2022.
- Minimize the multi-task training objective with a standard gradient-based algorithm.
- **[Auto-λ]** Liu, S., James, S., Davison, A. J., & Johns, E. [Auto-Lambda: Disentangling Dynamic Task Relationships](http://arxiv.org/abs/2202.03091). TMLR, 2022.
- **[Nash-MTL]** Navon, A., Shamsian, A., Achituve, I., Maron, H., Kawaguchi, K., Chechik, G., & Fetaya, E. [Multi-Task Learning as a Bargaining Game](http://arxiv.org/abs/2202.01017). ICML, 2022.
- **[Nash-MTL]** Navon, A., Shamsian, A., Achituve, I., Maron, H., Kawaguchi, K., Chechik, G., & Fetaya, E. [Multi-Task Learning as a Bargaining Game](http://arxiv.org/abs/2202.01017). ICML, 2022.
- Also resurrects important **Scale-invariant (SI)** baseline which minimizes $\sum_k \log \ell_k$.
- **[Rotograd]** Javaloy, A., & Valera, I. [RotoGrad: Gradient Homogenization in Multitask Learning](http://arxiv.org/abs/2103.02631). ICLR, 2022.
- **[RLW / RGW]** Lin, B., Ye, F., & Zhang, Y. [Reasonable Effectiveness of Random Weighting: A Litmus Test for Multi-Task Learning](http://arxiv.org/abs/2111.10603). TMLR, 2022.
- **[CAGrad]** Liu, B., Liu, X., Jin, X., Stone, P., & Liu, Q. [Conflict-Averse Gradient Descent for Multi-task Learning](https://arxiv.org/abs/2110.14048). NeurIPS, 2021.
Expand Down

0 comments on commit c846eb8

Please sign in to comment.