Toward a Standardized Representation for Deep Learning Collective Algorithms.

Benjamin Klenk

Jinsun Yoo William Won Meghan Cowan Nan Jiang Benjamin Klenk Srinivas Sridharan 0002 Tushar Krishna Toward a Standardized Representation for Deep Learning Collective Algorithms. 46-55 2025 March - April 45 IEEE Micro 2 https://doi.org/10.1109/MM.2025.3547363 db/journals/micro/micro45.html#YooWCJKSK25 streams/journals/micro

Zhenguo Wu Benjamin Klenk Larry Dennison Keren Bergman ACTINA: Adapting Circuit-Switching Techniques for AI Networking Architectures. 1211-1222 2025 SC https://doi.org/10.1145/3712285.3759842 conf/sc/2025 db/conf/sc/sc2025.html#WuKDB25 Jinsun Yoo William Won Meghan Cowan Nan Jiang Benjamin Klenk Srinivas Sridharan 0002 Tushar Krishna Towards a Standardized Representation for Deep Learning Collective Algorithms. 33-36 2024 HOTI https://doi.org/10.1109/HOTI63208.2024.00017 conf/hoti/2024 db/conf/hoti/hoti2024.html#YooWCJKSK24 streams/conf/hoti

Jinsun Yoo William Won Meghan Cowan Nan Jiang Benjamin Klenk Srinivas Sridharan 0002 Tushar Krishna Towards a Standardized Representation for Deep Learning Collective Algorithms. 2024 abs/2408.11008 CoRR https://doi.org/10.48550/arXiv.2408.11008 db/journals/corr/corr2408.html#abs-2408-11008 streams/journals/corr

George Michelogiannakis Benjamin Klenk Brandon Cook 0001 Min Yee Teh Madeleine Glick Larry Dennison Keren Bergman John Shalf A Case For Intra-rack Resource Disaggregation in HPC. 29:1-29:26 2022 19 ACM Trans. Archit. Code Optim. 2 https://doi.org/10.1145/3514245 db/journals/taco/taco19.html#Michelogiannakis22

Mehrdad Khani Shirkoohi Manya Ghobadi Mohammad Alizadeh Ziyi Zhu Madeleine Glick Keren Bergman Amin Vahdat Benjamin Klenk Eiman Ebrahimi SiP-ML: high-bandwidth optical network interconnects for machine learning training. 657-675 2021 SIGCOMM https://doi.org/10.1145/3452296.3472900 conf/sigcomm/2021 db/conf/sigcomm/sigcomm2021.html#ShirkoohiGAZGBV21 Benjamin Klenk Nan Jiang 0009 Greg Thorson Larry Dennison An In-Network Architecture for Accelerating Shared-Memory Multiprocessor Collectives. 996-1009 2020 ISCA https://doi.org/10.1109/ISCA45697.2020.00085 conf/isca/2020 db/conf/isca/isca2020.html#KlenkJTD20 Benjamin Klenk Larry Dennison Why Data Science and Machine Learning Need Silicon Photonics. 1-3 2020 OFC https://ieeexplore.ieee.org/document/9083135 conf/ofc/2020 db/conf/ofc/ofc2020.html#KlenkD30 Benjamin Klenk Communication architectures for scalable GPU-centric computing systems. 2018 1-207 University of Heidelberg, Germany http://www.ub.uni-heidelberg.de/archiv/23954 https://nbn-resolving.org/urn:nbn:de:bsz:16-heidok-239547 https://d-nb.info/1151544981 https://d-nb.info/1177691078 Benjamin Klenk Holger Fröning Hans Eberle Larry Dennison Relaxations for High-Performance Message Passing on Massively Parallel SIMT Processors. 855-865 2017 IPDPS https://doi.org/10.1109/IPDPS.2017.94 https://doi.ieeecomputersociety.org/10.1109/IPDPS.2017.94 conf/ipps/2017 db/conf/ipps/ipdps2017.html#KlenkFED17 Benjamin Klenk Holger Fröning An Overview of MPI Characteristics of Exascale Proxy Applications. 217-236 2017 ISC https://doi.org/10.1007/978-3-319-58667-0_12 conf/supercomputer/2017 db/conf/supercomputer/isc2017.html#KlenkF17

Lena Oden Benjamin Klenk Holger Fröning Analyzing GPU-controlled communication with dynamic parallelism in terms of performance and energy. 125-134 2016 57 Parallel Comput. https://doi.org/10.1016/j.parco.2016.02.005 db/journals/pc/pc57.html#OdenKF16

Benjamin Klenk Lena Oden Holger Fröning Analyzing communication models for distributed thread-collaborative processors in terms of energy and time. 318-327 2015 ISPASS https://doi.org/10.1109/ISPASS.2015.7095817 https://doi.ieeecomputersociety.org/10.1109/ISPASS.2015.7095817 conf/ispass/2015 db/conf/ispass/ispass2015.html#KlenkOF15 Lena Oden Benjamin Klenk Holger Fröning Energy-Efficient Collective Reduce and Allreduce Operations on Distributed GPUs. 483-492 2014 CCGRID https://doi.org/10.1109/CCGrid.2014.21 https://doi.ieeecomputersociety.org/10.1109/CCGrid.2014.21 conf/ccgrid/2014 db/conf/ccgrid/ccgrid2014.html#OdenKF14 Benjamin Klenk Lena Oden Holger Fröning Analyzing Put/Get APIs for Thread-Collaborative Processors. 411-418 2014 ICPP Workshops https://doi.org/10.1109/ICPPW.2014.61 https://doi.ieeecomputersociety.org/10.1109/ICPPW.2014.61 conf/icppw/2014 db/conf/icppw/icppw2014.html#KlenkOF14 Lena Oden Benjamin Klenk Holger Fröning Energy-efficient stencil computations on distributed GPUs using dynamic parallelism and GPU-controlled communication. 31-40 2014 E2SC@SC https://doi.org/10.1109/E2SC.2014.14 https://doi.ieeecomputersociety.org/10.1109/E2SC.2014.14 http://dl.acm.org/citation.cfm?id=2689714 conf/sc/2014e2sc db/conf/sc/e2sc2014.html#OdenKF14 Mohammad Alizadeh Keren Bergman Brandon Cook 0001 Meghan Cowan Larry Dennison Hans Eberle Eiman Ebrahimi Holger Fröning Manya Ghobadi Madeleine Glick Nan Jiang Nan Jiang 0009 Tushar Krishna George Michelogiannakis Lena Oden John Shalf Mehrdad Khani Shirkoohi Srinivas Sridharan 0002 Min Yee Teh Greg Thorson Amin Vahdat William Won Zhenguo Wu Jinsun Yoo Ziyi Zhu