{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T05:03:26Z","timestamp":1750309406304,"version":"3.41.0"},"reference-count":82,"publisher":"Association for Computing Machinery (ACM)","issue":"7","license":[{"start":{"date-parts":[[2024,9,27]],"date-time":"2024-09-27T00:00:00Z","timestamp":1727395200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/summer-heart-0930.chufeiyun1688.workers.dev:443\/https\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"NSF","award":["CCF-2047682, CCF-2008905, CCF-2146443, CNS-2235137, CPS-2230969, CNS-2300525, CNS-2343653, and CNS-2312397"],"award-info":[{"award-number":["CCF-2047682, CCF-2008905, CCF-2146443, CNS-2235137, CPS-2230969, CNS-2300525, CNS-2343653, and CNS-2312397"]}]},{"name":"NSF graduate research fellowship program; and Eugene McDermott Graduate Fellowship","award":["202006"],"award-info":[{"award-number":["202006"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Softw. Eng. Methodol."],"published-print":{"date-parts":[[2024,9,30]]},"abstract":"<jats:p>Natural language processing (NLP) has gained widespread adoption in the development of real-world applications. However, the black-box nature of neural networks in NLP applications poses a challenge when evaluating their performance, let alone ensuring it. Recent research has proposed testing techniques to enhance the trustworthiness of NLP-based applications. However, most existing works use a single, aggregated metric (i.e., accuracy) which is difficult for users to assess NLP model performance on fine-grained aspects, such as LCs. To address this limitation, we present ALiCT, an automated testing technique for validating NLP applications based on their LCs. ALiCT takes user-specified LCs as inputs and produces diverse test suite with test oracles for each of given LC. We evaluate ALiCT on two widely adopted NLP tasks, sentiment analysis and hate speech detection, in terms of diversity, effectiveness, and consistency. Using Self-BLEU and syntactic diversity metrics, our findings reveal that ALiCT generates test cases that are 190% and 2213% more diverse in semantics and syntax, respectively, compared to those produced by state-of-the-art techniques. In addition, ALiCT is capable of producing a larger number of NLP model failures in 22 out of 25 LCs over the two NLP applications.<\/jats:p>","DOI":"10.1145\/3672455","type":"journal-article","created":{"date-parts":[[2024,6,14]],"date-time":"2024-06-14T18:12:34Z","timestamp":1718388754000},"page":"1-33","update-policy":"https:\/\/summer-heart-0930.chufeiyun1688.workers.dev:443\/https\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Automated Testing Linguistic Capabilities of NLP Models"],"prefix":"10.1145","volume":"33","author":[{"ORCID":"https:\/\/summer-heart-0930.chufeiyun1688.workers.dev:443\/https\/orcid.org\/0009-0001-1756-5767","authenticated-orcid":false,"given":"Jaeseong","family":"Lee","sequence":"first","affiliation":[{"name":"The University of Texas at Dallas, Richardson, TX, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/summer-heart-0930.chufeiyun1688.workers.dev:443\/https\/orcid.org\/0000-0001-5035-3398","authenticated-orcid":false,"given":"Simin","family":"Chen","sequence":"additional","affiliation":[{"name":"The University of Texas at Dallas, Richardson, TX, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/summer-heart-0930.chufeiyun1688.workers.dev:443\/https\/orcid.org\/0000-0003-3031-8848","authenticated-orcid":false,"given":"Austin","family":"Mordahl","sequence":"additional","affiliation":[{"name":"The University of Texas at Dallas, Richardson, TX, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/summer-heart-0930.chufeiyun1688.workers.dev:443\/https\/orcid.org\/0009-0001-6775-1269","authenticated-orcid":false,"given":"Cong","family":"Liu","sequence":"additional","affiliation":[{"name":"University of California, Riverside, CA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/summer-heart-0930.chufeiyun1688.workers.dev:443\/https\/orcid.org\/0000-0002-5338-7347","authenticated-orcid":false,"given":"Wei","family":"Yang","sequence":"additional","affiliation":[{"name":"The University of Texas at Dallas, Richardson, TX, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/summer-heart-0930.chufeiyun1688.workers.dev:443\/https\/orcid.org\/0000-0002-2826-1857","authenticated-orcid":false,"given":"Shiyi","family":"Wei","sequence":"additional","affiliation":[{"name":"The University of Texas at Dallas, Richardson, TX, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2024,9,27]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D18-1316"},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2021.3136169"},{"key":"e_1_3_2_4_2","volume-title":"Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC\u201910)","author":"Baccianella Stefano","year":"2010","unstructured":"Stefano Baccianella, Andrea Esuli, and Fabrizio Sebastiani. 2010. SentiWordNet 3.0: An Enhanced Lexical Resource for Sentiment Analysis and Opinion Mining. In Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC\u201910). European Language Resources Association (ELRA), Valletta, Malta. Retrieved from https:\/\/summer-heart-0930.chufeiyun1688.workers.dev:443\/http\/www.lrec-conf.org\/proceedings\/lrec2010\/pdf\/769_Paper.pdf"},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1145\/3324884.3416609"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10439-023-03171-8"},{"key":"e_1_3_2_7_2","unstructured":"Calculator.net. 2023. Sample Size Calculator. Retrieved from https:\/\/summer-heart-0930.chufeiyun1688.workers.dev:443\/https\/www.calculator.net\/sample-size-calculator.html"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1145\/3368089.3409733"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1145\/3551349.3561158"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1145\/3540250.3549102"},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.01493"},{"key":"e_1_3_2_12_2","first-page":"240:1","article-title":"PaLM: Scaling Language Modeling with Pathways","volume":"24","author":"Chowdhery Aakanksha","year":"2023","unstructured":"Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, Parker Schuh, Kensen Shi, Sasha Tsvyashchenko, Joshua Maynez, Abhishek Rao, Parker Barnes, Yi Tay, Noam Shazeer, Vinodkumar Prabhakaran, Emily Reif, Nan Du, Ben Hutchinson, Reiner Pope, James Bradbury, Jacob Austin, Michael Isard, Guy Gur-Ari, Pengcheng Yin, Toju Duke, Anselm Levskaya, Sanjay Ghemawat, Sunipa Dev, Henryk Michalewski, Xavier Garcia, Vedant Misra, Kevin Robinson, Liam Fedus, Denny Zhou, Daphne Ippolito, David Luan, Hyeontaek Lim, Barret Zoph, Alexander Spiridonov, Ryan Sepassi, David Dohan, Shivani Agrawal, Mark Omernick, Andrew M. Dai, Thanumalayan Sankaranarayana Pillai, Marie Pellat, Aitor Lewkowycz, Erica Moreira, Rewon Child, Oleksandr Polozov, Katherine Lee, Zongwei Zhou, Xuezhi Wang, Brennan Saeta, Mark Diaz, Orhan Firat, Michele Catasta, Jason Wei, Kathy Meier-Hellstern, Douglas Eck, Jeff Dean, Slav Petrov, and Noah Fiedel. 2023. PaLM: Scaling Language Modeling with Pathways. Journal of Machine Learning Research 24, 240:1\u2013240:113. Retrieved from https:\/\/summer-heart-0930.chufeiyun1688.workers.dev:443\/http\/jmlr.org\/papers\/v24\/22-1144.html","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_3_2_13_2","series-title":"Proceedings of Machine Learning Research","first-page":"4455","volume-title":"Proceedings of the 39th International Conference on Machine Learning","volume":"162","author":"Cuadros Xavier Suau","year":"2022","unstructured":"Xavier Suau Cuadros, Luca Zappella, and Nicholas Apostoloff. 2022. Self-Conditioning Pre-Trained Language Models. In Proceedings of the 39th International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 162. Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato (Eds.). PMLR, 4455\u20134473. DOI: https:\/\/summer-heart-0930.chufeiyun1688.workers.dev:443\/https\/proceedings.mlr.press\/v162\/cuadros22a.html"},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.18653\/V1\/N19-1423"},{"key":"e_1_3_2_15_2","first-page":"370","volume-title":"Introduction to Statistical Analysis","author":"Dixon W. J.","year":"1951","unstructured":"W. J. Dixon and Frank J. Massey. 1951. Introduction to Statistical Analysis. McGraw-Hill, New York, NY, 370."},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1145\/3338906.3338954"},{"key":"e_1_3_2_17_2","first-page":"653","volume-title":"Proceedings of the 27th International Conference on Computational Linguistics","author":"Ebrahimi Javid","year":"2018","unstructured":"Javid Ebrahimi, Daniel Lowd, and Dejing Dou. 2018a. On Adversarial Examples for Character-Level Neural Machine Translation. In Proceedings of the 27th International Conference on Computational Linguistics. Association for Computational Linguistics, Santa Fe, New Mexico, USA, 653\u2013663. Retrieved from https:\/\/summer-heart-0930.chufeiyun1688.workers.dev:443\/https\/aclanthology.org\/C18-1055"},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P18-2006"},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1145\/3293882.3330566"},{"key":"e_1_3_2_20_2","first-page":"1243","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Gehring Jonas","year":"2017","unstructured":"Jonas Gehring, Michael Auli, David Grangier, Denis Yarats, and Yann N. Dauphin. 2017. Convolutional Sequence to Sequence Learning. In Proceedings of the International Conference on Machine Learning. PMLR, 1243\u20131252."},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1145\/3243734.3243792"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.1145\/3368089.3409756"},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1145\/3368089.3409754"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1145\/3377811.3380339"},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE43902.2021.00047"},{"key":"e_1_3_2_26_2","unstructured":"Chaitra V. Hegde and Shrikumar Patil. 2020. Unsupervised Paraphrase Generation Using Pre-Trained Language Models. arXiv:2006.05477. Retrieved from https:\/\/summer-heart-0930.chufeiyun1688.workers.dev:443\/https\/arxiv.org\/abs\/2006.05477"},{"key":"e_1_3_2_27_2","unstructured":"Matthew Honnibal and Ines Montani. 2017. spaCy 2: Natural Language Understanding with Bloom Embeddings Convolutional Neural Networks and Incremental Parsing."},{"key":"e_1_3_2_28_2","unstructured":"HuggingFace. 2022. HuggingFace. Retrieved from https:\/\/summer-heart-0930.chufeiyun1688.workers.dev:443\/https\/huggingface.co"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1145\/3460319.3464825"},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11704-019-9094-0"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.18653\/V1\/E17-2068"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2019.00108"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P19-1340"},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P18-1249"},{"key":"e_1_3_2_35_2","article-title":"Understanding the Fatal Tesla Accident on Autopilot and the NHTSA Probe","author":"Lambert Fred","year":"2016","unstructured":"Fred Lambert. 2016. Understanding the Fatal Tesla Accident on Autopilot and the NHTSA Probe. Electrek, July 1.","journal-title":"Electrek"},{"key":"e_1_3_2_36_2","article-title":"Tesla Fatal Crash: \u2019Autopilot\u2019 Mode sped up car Before Driver Killed, Report Finds","author":"Levin Sam","year":"2018","unstructured":"Sam Levin. 2018. Tesla Fatal Crash: \u2019Autopilot\u2019 Mode sped up car Before Driver Killed, Report Finds. The Guardian, June 8.","journal-title":"The Guardian"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.14722\/ndss.2019.23138"},{"key":"e_1_3_2_38_2","doi-asserted-by":"crossref","unstructured":"Linyang Li Ruotian Ma Qipeng Guo Xiangyang Xue and Xipeng Qiu. 2020a. BERT-ATTACK: Adversarial Attack Against BERT Using BERT. arXiv:2004.09984. Retrieved from https:\/\/summer-heart-0930.chufeiyun1688.workers.dev:443\/https\/arxiv.org\/abs\/2004.09984","DOI":"10.18653\/v1\/2020.emnlp-main.500"},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.emnlp-main.500"},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10462-021-09973-3"},{"key":"e_1_3_2_41_2","unstructured":"Yinhan Liu Myle Ott Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy Mike Lewis Luke Zettlemoyer and Veselin Stoyanov. 2019. RoBERTa: A Robustly Optimized BERT Pretraining Approach. arXiv:1907.11692. Retrieved from https:\/\/summer-heart-0930.chufeiyun1688.workers.dev:443\/http\/arxiv.org\/abs\/1907.11692"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1145\/3238147.3238202"},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISSRE.2018.00021"},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.24963\/ijcai.2020\/64"},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.5555\/972470.972475"},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v35i17.17745"},{"key":"e_1_3_2_47_2","series-title":"Workshop Track Proceedings","volume-title":"Proceedings of the 1st International Conference on Learning Representations (ICLR\u201913)","author":"Mikolov Tom\u00e1s","year":"2013","unstructured":"Tom\u00e1s Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient Estimation of Word Representations in Vector Space. In Proceedings of the 1st International Conference on Learning Representations (ICLR\u201913), Workshop Track Proceedings. Yoshua Bengio and Yann LeCun (Eds.). Retrieved from https:\/\/summer-heart-0930.chufeiyun1688.workers.dev:443\/http\/arxiv.org\/abs\/1301.3781"},{"key":"e_1_3_2_48_2","unstructured":"John X. Morris Eli Lifland Jin Yong Yoo and Yanjun Qi. 2020. TextAttack: A Framework for Adversarial Attacks in Natural Language Processing. arXiv:2005.05909. Retrieved from https:\/\/summer-heart-0930.chufeiyun1688.workers.dev:443\/https\/arxiv.org\/abs\/2005.05909"},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","unstructured":"OpenAI. 2023a. GPT-4 Technical Report. arXiv:2303.08774. DOI: 10.48550\/ARXIV.2303.08774","DOI":"10.48550\/ARXIV.2303.08774"},{"key":"e_1_3_2_50_2","unstructured":"OpenAI. 2023b. GPT Model Documentation. Retrieved from https:\/\/summer-heart-0930.chufeiyun1688.workers.dev:443\/https\/platform.openai.com\/docs\/introduction"},{"key":"e_1_3_2_51_2","doi-asserted-by":"publisher","DOI":"10.3115\/1073083.1073135"},{"key":"e_1_3_2_52_2","doi-asserted-by":"publisher","DOI":"10.1145\/3132747.3132785"},{"key":"e_1_3_2_53_2","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/D14-1162"},{"key":"e_1_3_2_54_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2019.00107"},{"key":"e_1_3_2_55_2","unstructured":"Alec Radford Karthik Narasimhan Tim Salimans and Ilya Sutskever. 2018. Improving Language Understanding by Generative Pre-Training. Retrieved from https:\/\/summer-heart-0930.chufeiyun1688.workers.dev:443\/https\/api.semanticscholar.org\/CorpusID:49313245"},{"issue":"8","key":"e_1_3_2_56_2","first-page":"9","article-title":"Language Models are Unsupervised Multitask Learners","volume":"1","author":"Radford Alec","year":"2019","unstructured":"Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language Models are Unsupervised Multitask Learners. OpenAI Blog 1, 8 (2019), 9. Retrieved from https:\/\/summer-heart-0930.chufeiyun1688.workers.dev:443\/https\/api.semanticscholar.org\/CorpusID:160025533","journal-title":"OpenAI Blog"},{"key":"e_1_3_2_57_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P19-1103"},{"key":"e_1_3_2_58_2","unstructured":"Marco Tulio Ribeiro. 2023. CHECKLIST Github Repository. Retrieved from https:\/\/summer-heart-0930.chufeiyun1688.workers.dev:443\/https\/github.com\/marcotcr\/checklist\/tree\/master"},{"key":"e_1_3_2_59_2","doi-asserted-by":"publisher","DOI":"10.1145\/2939672.2939778"},{"key":"e_1_3_2_60_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.442"},{"key":"e_1_3_2_61_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.woah-1.15"},{"key":"e_1_3_2_62_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.acl-long.4"},{"key":"e_1_3_2_63_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W17-1101"},{"key":"e_1_3_2_64_2","doi-asserted-by":"publisher","DOI":"10.1109\/78.650093"},{"key":"e_1_3_2_65_2","first-page":"1631","volume-title":"Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing","author":"Socher Richard","year":"2013","unstructured":"Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Ng, and Christopher Potts. 2013. Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Seattle, WA, USA, 1631\u20131642. DOI: https:\/\/summer-heart-0930.chufeiyun1688.workers.dev:443\/https\/aclanthology.org\/D13-1170"},{"key":"e_1_3_2_66_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2022.3141758"},{"key":"e_1_3_2_67_2","doi-asserted-by":"publisher","DOI":"10.1145\/3377811.3380420"},{"key":"e_1_3_2_68_2","first-page":"3104","article-title":"Sequence to Sequence Learning with Neural Networks","volume":"27","author":"Sutskever Ilya","year":"2014","unstructured":"Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014. Sequence to Sequence Learning with Neural Networks. Advances in Neural Information Processing Systems 27, 3104\u20133112.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_69_2","doi-asserted-by":"publisher","DOI":"10.1145\/3180155.3180220"},{"key":"e_1_3_2_70_2","doi-asserted-by":"publisher","unstructured":"Hugo Touvron Thibaut Lavril Gautier Izacard Xavier Martinet Marie-Anne Lachaux Timoth\u00e9e Lacroix Baptiste Rozi\u00e9re Naman Goyal Eric Hambro Faisal Azhar Aur\u00e9lien Rodriguez Armand Joulin Edouard Grave and Guillaume Lample. 2023. LLaMA: Open and Efficient Foundation Language Models. arXiv:2302.13971. DOI: 10.48550\/ARXIV.2302.13971","DOI":"10.48550\/ARXIV.2302.13971"},{"key":"e_1_3_2_71_2","doi-asserted-by":"publisher","DOI":"10.1145\/3238147.3238165"},{"key":"e_1_3_2_72_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2019.2953066"},{"key":"e_1_3_2_73_2","first-page":"5998","article-title":"Attention is All You Need","volume":"30","author":"Vaswani Ashish","year":"2017","unstructured":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is All You Need. Advances in Neural Information Processing Systems 30, 5998\u20136008.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_74_2","doi-asserted-by":"publisher","DOI":"10.1145\/3293882.3330579"},{"key":"e_1_3_2_75_2","doi-asserted-by":"publisher","DOI":"10.1145\/3368089.3409671"},{"key":"e_1_3_2_76_2","doi-asserted-by":"publisher","DOI":"10.1109\/SANER53432.2022.00056"},{"key":"e_1_3_2_77_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.findings-emnlp.135"},{"key":"e_1_3_2_78_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.540"},{"key":"e_1_3_2_79_2","series-title":"Proceedings of Machine Learning Research","first-page":"422","volume-title":"Proceedings of the 23rd International Conference on Artificial Intelligence and Statistics (AISTATS\u201920)","volume":"108","author":"Zhang Ruiyi","year":"2020","unstructured":"Ruiyi Zhang, Changyou Chen, Zhe Gan, Zheng Wen, Wenlin Wang, and Lawrence Carin. 2020. Nested-Wasserstein Self-Imitation Learning for Sequence Generation. In Proceedings of the 23rd International Conference on Artificial Intelligence and Statistics (AISTATS\u201920), Proceedings of Machine Learning Research, Vol. 108. Silvia Chiappa and Roberto Calandra (Eds.). PMLR, 422\u2013433. Retrieved from https:\/\/summer-heart-0930.chufeiyun1688.workers.dev:443\/http\/proceedings.mlr.press\/v108\/zhang20b.html"},{"key":"e_1_3_2_80_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.acl-long.153"},{"key":"e_1_3_2_81_2","doi-asserted-by":"publisher","DOI":"10.1109\/RBME.2022.3210270"},{"key":"e_1_3_2_82_2","doi-asserted-by":"publisher","DOI":"10.1145\/3209978.3210080"},{"key":"e_1_3_2_83_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.319"}],"container-title":["ACM Transactions on Software Engineering and Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/summer-heart-0930.chufeiyun1688.workers.dev:443\/https\/dl.acm.org\/doi\/10.1145\/3672455","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/summer-heart-0930.chufeiyun1688.workers.dev:443\/https\/dl.acm.org\/doi\/pdf\/10.1145\/3672455","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T00:58:01Z","timestamp":1750294681000},"score":1,"resource":{"primary":{"URL":"https:\/\/summer-heart-0930.chufeiyun1688.workers.dev:443\/https\/dl.acm.org\/doi\/10.1145\/3672455"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,9,27]]},"references-count":82,"journal-issue":{"issue":"7","published-print":{"date-parts":[[2024,9,30]]}},"alternative-id":["10.1145\/3672455"],"URL":"https:\/\/summer-heart-0930.chufeiyun1688.workers.dev:443\/https\/doi.org\/10.1145\/3672455","relation":{},"ISSN":["1049-331X","1557-7392"],"issn-type":[{"type":"print","value":"1049-331X"},{"type":"electronic","value":"1557-7392"}],"subject":[],"published":{"date-parts":[[2024,9,27]]},"assertion":[{"value":"2023-09-07","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-05-09","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-09-27","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}