Recently, the paper of Kewei Tu's research team titled Do PLMs Know and Understand Ontological Knowledge? was honored with the Outstanding Paper Award at the 61st Annual Conference of the International Association for Computational Linguistics (ACL) held in Toronto, Canada. Weiqi Wu, graduate of 2023, is the first author. Chengyue Jiang, PhD student from the class of 2021, is the second author. Alibaba Damo Academy is the collaborating institution. Kewei Tu is the corresponding author.
The researchers propose to explore the extent to which pretrained language models (PLMs) know and understand ontological knowledge, bringing new advances in the systematic analysis of pretrained language models. They probe the ability of PLMs to memorize ontological knowledge, which involves types of entities, hierarchical relationships among classes and properties, as well as domain and range constraints of properties. In order to investigate whether the models further understand the knowledge, they comprehensively study whether they can reliably perform logical reasoning with given knowledge according to ontological entailment rules.
The findings of their experiments reveal that encoder-only models like BERT and RoBERTa do exhibit some capability to encode ontological knowledge and can reason with the memorized knowledge. However, there are still limitations in their abilities to both memorize and fully understand ontological knowledge. On the other hand, the team's exploration into the decoder-only model, ChatGPT, demonstrates a noteworthy improvement in both aspects, signifying the potential for further advancements.
ACL, which stands for Annual Meeting of the Association for Computational Linguistics, is organized by the Association for Computational Linguistics and holds a distinguished position as the most influential international academic conference in the field of computational linguistics and natural language processing. This year, the conference received nearly 5,000 submissions, with an acceptance rate of 20.7%. Including the award-winning paper, Kewei Tu’s research team has published 6 papers at the main conference and 2 papers at the Findings category in ACL 2023, covering various topics such as syntactic parsing, information extraction, language models, and basic model architectures. In addition, the team participated in the paper titled DAMO-NLP at SemEval-2023 Task 2: A Unified Retrieval-Augmented System for Multilingual Named Entity Recognition, which was presented by Alibaba Damo Academy. The paper won the Best System Paper Award at the ACL 2023 workshop SemEval (International Workshop on Semantic Evaluation).
Author Biography
Weiqi Wu: first author of the paper. She was an undergraduate student of the class of 2019 from the School of Information Science and Technology, ShanghaiTech University. Throughout her undergraduate years, she passionately pursued research in the fields of artificial intelligence and natural language processing under the guidance of Professor Kewei Tu, with knowledge acquisition and large language models as her main research interest. She has published multiple papers at top conferences in the field of natural language processing, such as ACL and EMNLP. Her achievements were recognized with the 2023 ShanghaiTech President’s Award and the Shanghai Outstanding Graduate Student Award.
Chengyue Jiang: second author of the paper. He is currently a PhD student at the School of Information Science and Technology, ShanghaiTech University, under the mentorship of Professor Kewei Tu. He was an undergraduate student of the class of 2015 at the School of Information Science and Technology. Chengyue's research centers on the combination of symbolic knowledge and neural network models, information extraction, and large-scale pretrained language models. His research has been acknowledged with the publication of several papers at top international conferences in natural language processing, including ACL, EMNLP, and EACL.
Personal Homepage: https://jeffchy.github.io/
Kewei Tu: corresponding author of the paper. He is a tenured associate professor with the School of Information Science and Technology at ShanghaiTech University. His research areas include natural language processing, machine learning, and artificial intelligence in general. He has almost 100 publications in major conferences and journals including ACL, EMNLP, NAACL, AAAI, etc. He served as PC members and area chairs at several NLP and AI conferences and as an action editor of ACL Rolling Review.
Personal Homepage: http://faculty.sist.shanghaitech.edu.cn/faculty/tukw/
沪公网安备 31011502006855号
