MTrainS: Improving DLRM training efficiency using heterogeneous memories.

Andrew Tulloch

Hiwot Tadese Kassa Paul Johnson Jason Akers Mrinmoy Ghosh Andrew Tulloch Dheevatsa Mudigere Jongsoo Park Xing Liu Ronald G. Dreslinski Ehsan K. Ardestani MTrainS: Improving DLRM training efficiency using heterogeneous memories. 2023 abs/2305.01515 CoRR https://doi.org/10.48550/arXiv.2305.01515 db/journals/corr/corr2305.html#abs-2305-01515

Dheevatsa Mudigere Yuchen Hao Jianyu Huang Zhihao Jia Andrew Tulloch Srinivas Sridharan 0002 Xing Liu Mustafa Ozdal Jade Nie Jongsoo Park Liang Luo Jie Amy Yang Leon Gao Dmytro Ivchenko Aarti Basant Yuxi Hu 0001 Jiyan Yang Ehsan K. Ardestani Xiaodong Wang 0020 Rakesh Komuravelli Ching-Hsiang Chu Serhat Yilmaz Huayu Li Jiyuan Qian Zhuobo Feng Yinbin Ma Junjie Yang 0005 Ellie Wen Hong Li Lin Yang Chonglin Sun Whitney Zhao Dimitry Melts Krishna Dhulipala K. R. Kishore Tyler Graf Assaf Eisenman Kiran Kumar Matam Adi Gangidi Guoqiang Jerry Chen Manoj Krishnan Avinash Nayak Krishnakumar Nair Bharath Muthiah Mahmoud khorashadi Pallab Bhattacharya Petr Lapukhov Maxim Naumov Ajit Mathews Lin Qiao Mikhail Smelyanskiy Bill Jia Vijay Rao Software-hardware co-design for fast and scalable training of deep learning recommendation models. 993-1011 2022 ISCA https://doi.org/10.1145/3470496.3533727 https://www.wikidata.org/entity/Q131025637 conf/isca/2022 db/conf/isca/isca2022.html#MudigereHHJT0LO22

Dheevatsa Mudigere Yuchen Hao Jianyu Huang Andrew Tulloch Srinivas Sridharan 0002 Xing Liu Mustafa Ozdal Jade Nie Jongsoo Park Liang Luo Jie Amy Yang Leon Gao Dmytro Ivchenko Aarti Basant Yuxi Hu 0001 Jiyan Yang Ehsan K. Ardestani Xiaodong Wang 0020 Rakesh Komuravelli Ching-Hsiang Chu Serhat Yilmaz Huayu Li Jiyuan Qian Zhuobo Feng Yinbin Ma Junjie Yang 0005 Ellie Wen Hong Li Lin Yang Chonglin Sun Whitney Zhao Dimitry Melts Krishna Dhulipala K. R. Kishore Tyler Graf Assaf Eisenman Kiran Kumar Matam Adi Gangidi Guoqiang Jerry Chen Manoj Krishnan Avinash Nayak Krishnakumar Nair Bharath Muthiah Mahmoud khorashadi Pallab Bhattacharya Petr Lapukhov Maxim Naumov Lin Qiao Mikhail Smelyanskiy Bill Jia Vijay Rao High-performance, Distributed Training of Large-scale Deep Learning Recommendation Models. 2021 abs/2104.05158 CoRR https://arxiv.org/abs/2104.05158 db/journals/corr/corr2104.html#abs-2104-05158

Jie Amy Yang Jianyu Huang Jongsoo Park Ping Tak Peter Tang Andrew Tulloch Mixed-Precision Embedding Using a Cache. 2020 abs/2010.11305 CoRR https://arxiv.org/abs/2010.11305 db/journals/corr/corr2010.html#abs-2010-11305

Carole-Jean Wu David Brooks 0001 Kevin Chen Douglas Chen Sy Choudhury Marat Dukhan Kim M. Hazelwood Eldad Isaac Yangqing Jia Bill Jia Tommer Leyvand Hao Lu Yang Lu 0013 Lin Qiao Brandon Reagen Joe Spisak Fei Sun 0002 Andrew Tulloch Peter Vajda Xiaodong Wang 0020 Yanghan Wang Bram Wasti Yiming Wu 0013 Ran Xian Sungjoo Yoo Peizhao Zhang Machine Learning at Facebook: Understanding Inference at the Edge. 331-344 2019 HPCA https://doi.org/10.1109/HPCA.2019.00048 conf/hpca/2019 db/conf/hpca/hpca2019.html#WuBCCCDHIJJLLLQ19

Bing Xu Andrew Tulloch Yunpeng Chen Xiaomeng Yang Lin Qiao Hybrid Composition with IdleBlock: More Efficient Networks for Image Recognition. 2019 abs/1911.08609 CoRR http://arxiv.org/abs/1911.08609 db/journals/corr/corr1911.html#abs-1911-08609

Maxim Naumov Utku Diril Jongsoo Park Benjamin Ray Jedrzej Jablonski Andrew Tulloch On Periodic Functions as Regularizers for Quantization of Neural Networks. 2018 abs/1811.09862 CoRR http://arxiv.org/abs/1811.09862 db/journals/corr/corr1811.html#abs-1811-09862

Jongsoo Park Maxim Naumov Protonu Basu Summer Deng Aravind Kalaiah Daya Shanker Khudia James Law Parth Malani Andrey Malevich Nadathur Satish Juan Miguel Pino Martin Schatz Alexander Sidorov Viswanath Sivakumar Andrew Tulloch Xiaodong Wang 0020 Yiming Wu 0013 Hector Yuen Utku Diril Dmytro Dzhulgakov Kim M. Hazelwood Bill Jia Yangqing Jia Lin Qiao Vijay Rao Nadav Rotem Sungjoo Yoo Mikhail Smelyanskiy Deep Learning Inference in Facebook Data Centers: Characterization, Performance Optimizations and Hardware Implications. 2018 abs/1811.09886 CoRR http://arxiv.org/abs/1811.09886 db/journals/corr/corr1811.html#abs-1811-09886

Arthur Szlam Andrew Tulloch Mark Tygert Accurate Low-Rank Approximations Via a Few Iterations of Alternating Least Squares. 425-433 2017 38 SIAM J. Matrix Anal. Appl. 2 https://doi.org/10.1137/16M1064556 db/journals/siammax/siammax38.html#SzlamTT17

Priya Goyal Piotr Dollár Ross B. Girshick Pieter Noordhuis Lukasz Wesolowski Aapo Kyrola Andrew Tulloch Yangqing Jia Kaiming He Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour. 2017 abs/1706.02677 CoRR http://arxiv.org/abs/1706.02677 db/journals/corr/corr1706.html#GoyalDGNWKTJH17

Andrew Tulloch Yangqing Jia High performance ultra-low-precision convolutions on mobile devices. 2017 abs/1712.02427 CoRR http://arxiv.org/abs/1712.02427 db/journals/corr/corr1712.html#abs-1712-02427

Arthur Szlam Andrew Tulloch Mark Tygert Accurate principal component analysis via a few iterations of alternating least squares. 2016 abs/1603.01765 CoRR http://arxiv.org/abs/1603.01765 db/journals/corr/corr1603.html#SzlamTT16

Jason B. AkersJason Akers Ehsan K. Ardestani Aarti Basant Protonu Basu Pallab Bhattacharya David Brooks 0001 Douglas Chen Guoqiang Jerry Chen Kevin Chen Yunpeng Chen Sy Choudhury Ching-Hsiang Chu Summer Deng Krishna Dhulipala Utku Diril Piotr Dollár Ronald G. Dreslinski Marat Dukhan Dmytro Dzhulgakov Assaf Eisenman Zhuobo Feng Adi Gangidi Leon Gao Mrinmoy Ghosh Ross B. Girshick Priya Goyal Tyler Graf Yuchen Hao Kim M. Hazelwood Kaiming He Yuxi Hu 0001 Jianyu Huang Eldad Isaac Dmytro Ivchenko Jedrzej Jablonski Bill Jia Yangqing Jia Zhihao Jia Paul Johnson Aravind Kalaiah Hiwot KassaHiwot Tadese Kassa Mahmoud khorashadi Daya Shanker Khudia K. R. Kishore Rakesh Komuravelli Manoj Krishnan Aapo Kyrola Petr Lapukhov James Law Tommer Leyvand Hong Li Huayu Li Xing Liu Hao Lu Yang Lu 0013 Liang Luo Yinbin Ma Parth Malani Andrey Malevich Kiran Kumar Matam Ajit Mathews Dimitry Melts Dheevatsa Mudigere Bharath Muthiah Krishnakumar Nair Maxim Naumov Avinash Nayak Jade Nie Pieter Noordhuis Muhammet Mustafa OzdalMustafa Ozdal Jongsoo Park Juan Pino 0001Juan Miguel Pino Jiyuan Qian Lin Qiao Vijay Rao Benjamin Ray Brandon Reagen Nadav Rotem Nadathur Satish Martin Schatz Alexander Sidorov Viswanath Sivakumar Mikhail Smelyanskiy Joe Spisak Srinivas Sridharan 0002 Chonglin Sun Fei Sun 0002 Arthur Szlam Ping Tak Peter Tang Mark Tygert Peter Vajda Xiaodong Wang 0020 Yanghan Wang Bram Wasti Ellie Wen Lukasz Wesolowski Carole-Jean Wu Yiming Wu 0013 Ran Xian Bing Xu Jie Amy Yang Jiyan Yang Junjie Yang 0005 Lin Yang Xiaomeng Yang Serhat Yilmaz Sungjoo Yoo Hector Yuen Peizhao Zhang Whitney Zhao