Talks and presentations

Natural Language Processing and Representation Learning

November 02, 2022

Invited Talk, Frontier Forum on Intelligent Education, East China Normal University, Shanghai

Natural language typically refers to human language, which serves as a carrier of logical thinking, a means of communication, and a vehicle for cultural heritage. The processing of natural language is an important research content in artificial intelligence, often referred to as the pearl on the crown of AI. An essential foundational step in natural language processing is language representation learning, which aims to construct formal or mathematical descriptions of natural language so that it can be represented in computers and automatically processed by computer programs. Early language representation methods mainly used symbolic discrete representations. In recent years, deep neural networks have been widely applied in natural language processing, not only achieving performance surpassing traditional statistical methods in many tasks such as text classification, sequence labeling, machine translation, and automatic question answering, but also enabling end-to-end training, avoiding cumbersome feature engineering. The first part of this talk will introduce the basic tasks, application areas, research history, and technological development trends of natural language processing. The second part will introduce neural network-based language representation learning methods at various granularities such as words, phrases, sentences, and sentence pairs, explaining how the latent grammatical or semantic features of language are distributedly stored in a group of neurons and represented using dense, low-dimensional, continuous vectors. It will also discuss recent research trends in neural language representation learning from the perspectives of models and learning.

Interpretability Analysis of Artificial Intelligence - A Case Study in Natural Language Processing

July 22, 2022

Invited Talk, 2022 China Multimedia Conference, Guiyang, China

Interpretability in machine learning and deep learning refers to the extent to which model predictions can be explained in a way that is understandable and straightforward to the audience. In recent years, deep learning has achieved successful applications in natural language processing, significantly improving the performance of various tasks. However, due to its inherent complexity, the comprehensibility and interpretability are not satisfactory, which also hinders the further promotion of deep learning methods. This talk first introduces what interpretability analysis is, what interpretability analysis tasks exist in natural language processing, and the purpose of interpretability analysis. Then, it introduces the current development status of interpretability analysis in the field of natural language processing from three aspects: understanding the functional attributes of model components, explaining the behavior of model predictions, and model diagnosis. Finally, it discusses future research trends.

Incorporating Dynamic Structures into Pre-trained Language Models

July 22, 2022

Invited Talk, Workshop on Dynamic Neural Networks, International Conference on Machine Learning, Baltimore, USA

Recent years have witnessed the great success of large-scale pre-trained language models. However, performing the entire language model for each sample can be computationally uneconomical. Hence, dynamic networks are attracting a lot of attention in the NLP community, which can adapt their structures or parameters to the input samples during inference. In contrast to static language models, dynamic ones enjoy favorable properties such as efficiency, adaptiveness, accuracy, etc. In this talk, I will review recent advances in dynamic networks in NLP and discuss the prospects and challenges of applying dynamic structure to pre-trained language models.

Robustness Issues in Information Extraction

June 02, 2022

Invited talk, Beijing Zhiyuan Conference Natural Language Processing Forum, Beijing, China

Information extraction mainly includes two major tasks: named entity recognition and relation extraction, aiming to automatically extract key information from massive unstructured text, thereby effectively supporting downstream tasks such as knowledge graph construction and intelligent question answering. In the era of deep learning, since neural networks, particularly pre-trained models, can automatically extract high-level semantic features, more attention has been focused on how to construct pre-training tasks to achieve more comprehensive semantic knowledge embedding and how to efficiently use such models. However, the automatic feature extraction of deep learning models inevitably leads to shortcut learning problems, resulting in robustness deficiencies in real-world application scenarios, posing hidden dangers for downstream applications of information extraction, especially severe in low-resource environments. This talk will conduct an in-depth analysis of robustness issues in information extraction, explore the underlying reasons affecting model robustness, and introduce our research achievements in improving the robustness of information extraction models in scenarios such as weak samples, few-shot learning, unlabeled data, and cross-domain settings.

Intelligent Social Media Mining Based on Deep Learning

June 06, 2021

Invited Talk, 2021 Global Artificial Intelligence Technology Conference 'Development and Challenges' Intelligent Media Special Forum, Hangzhou, China

Social media refers to various media platforms through which people communicate in online society, possessing significant commercial and social value, and serving as important channels for information dissemination and maintaining social relationships. Over the past few years, the Natural Language Processing team at Fudan University has conducted various intelligent mining research on social media, forming a chain of social media understanding, discovery, and prediction. This includes understanding non-standard textual content on social media, discovering valuable information from social media, and predicting user behavior on social media. This talk mainly introduces methods for predicting user behavior on social media, including microblog tag recommendation, @user (company) recommendation, retweet behavior prediction, user topic participation prediction, expert recommendation, and incorporating multimodal information in social media mining.