From c846eb8c12372f5ec26cb1162693de4220c046e7 Mon Sep 17 00:00:00 2001 From: Manchery Date: Thu, 8 Jun 2023 00:16:01 +0800 Subject: [PATCH] Update README.md --- README.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index f2b71f9..d4a0e21 100644 --- a/README.md +++ b/README.md @@ -29,6 +29,7 @@ Please let me know if you find any mistakes or omissions! Your contribution is w - [Others](#others) - [Optimization](#optimization) - [Loss & Gradient Strategy](#loss--gradient-strategy) + - [Task Interference](#task-interference) - [Task Sampling](#task-sampling) - [Adversarial Training](#adversarial-training) - [Pareto](#pareto) @@ -244,13 +245,15 @@ Please let me know if you find any mistakes or omissions! Your contribution is w ### Loss & Gradient Strategy - **[MoCo]** Fernando, H. D., Shen, H., Liu, M., Chaudhury, S., Murugesan, K., & Chen, T. [Mitigating Gradient Bias in Multi-objective Learning: A Provably Convergent Approach](https://openreview.net/forum?id=dLAYGdKTi2). ICLR, 2023. +- **[FAMO]** Liu, B., Feng, Y., Stone, P., & Liu, Q. [FAMO: Fast Adaptive Multitask Optimization](http://arxiv.org/abs/2306.03792). ArXiv, 2023. - ✨ **[ForkMerge]** Jiang, J., Chen, B., Pan, J., Wang, X., Dapeng, L., Jiang, J., & Long, M. [ForkMerge: Overcoming Negative Transfer in Multi-Task Learning](http://arxiv.org/abs/2301.12618). ArXiv, 2023. - **[AuxiNash]** Shamsian, A., Navon, A., Glazer, N., Kawaguchi, K., Chechik, G., & Fetaya, E. [Auxiliary Learning as an Asymmetric Bargaining Game](https://arxiv.org/abs/2301.13501). ArXiv, 2023. - ✨ Xin, Derrick, Behrooz Ghorbani, Justin Gilmer, Ankush Garg, and Orhan Firat. **[Do Current Multi-Task Optimization Methods in Deep Learning Even Help?](https://openreview.net/forum?id=A2Ya5aLtyuG)** NeurIPS, 2022. - **[Unitary Scalarization]** Kurin, V., De Palma, A., Kostrikov, I., Whiteson, S., & Kumar, M. P. [In Defense of the Unitary Scalarization for Deep Multi-Task Learning](http://arxiv.org/abs/2201.04122). NeurIPS, 2022. - Minimize the multi-task training objective with a standard gradient-based algorithm. - **[Auto-λ]** Liu, S., James, S., Davison, A. J., & Johns, E. [Auto-Lambda: Disentangling Dynamic Task Relationships](http://arxiv.org/abs/2202.03091). TMLR, 2022. -- **[Nash-MTL]** Navon, A., Shamsian, A., Achituve, I., Maron, H., Kawaguchi, K., Chechik, G., & Fetaya, E. [Multi-Task Learning as a Bargaining Game](http://arxiv.org/abs/2202.01017). ICML, 2022. +- **[Nash-MTL]** Navon, A., Shamsian, A., Achituve, I., Maron, H., Kawaguchi, K., Chechik, G., & Fetaya, E. [Multi-Task Learning as a Bargaining Game](http://arxiv.org/abs/2202.01017). ICML, 2022. + - Also resurrects important **Scale-invariant (SI)** baseline which minimizes $\sum_k \log \ell_k$. - **[Rotograd]** Javaloy, A., & Valera, I. [RotoGrad: Gradient Homogenization in Multitask Learning](http://arxiv.org/abs/2103.02631). ICLR, 2022. - **[RLW / RGW]** Lin, B., Ye, F., & Zhang, Y. [Reasonable Effectiveness of Random Weighting: A Litmus Test for Multi-Task Learning](http://arxiv.org/abs/2111.10603). TMLR, 2022. - **[CAGrad]** Liu, B., Liu, X., Jin, X., Stone, P., & Liu, Q. [Conflict-Averse Gradient Descent for Multi-task Learning](https://arxiv.org/abs/2110.14048). NeurIPS, 2021.