Zhengxiao Du

Zhengxiao Du

PhD Student

Tsinghua University

Biography

I am a PhD student in the Department of Computer Science and Technology at Tsinghua University. I am also a member of the Knowledge Engineering Group, advised by Prof. Jie Tang.

My research interest lies in the application of machine learning algorithms to real-world systems, including information retrieval [ECML/PKDD'18,KDD'19,TKDE,SIGIR'19], knowledge graphs [TKDE], and pretrained language models[ACL'21,ACL'21]. Generally, I am also interested in the algorithms for machine learning and reinforcement learning.

I am currently co-leading the pre-training team of ChatGLM group at ZhipuAI.

In my spare time, I like to read science fiction novels and history books.

Interests
  • Data Mining
  • Recommender System
  • Knowledge Graph
Education

  • PhD in Computer Science, 2020 - Now

    Tsinghua University

  • B.E. in Computer Science, 2016 - 2020

    Tsinghua University

Recent News

[26/09/24] Our paper Understanding Emergent Abilities of Language Models from the Loss Perspective is accepted at NeurIPS 2024.

[05/06/24] We release GLM-4-9B, with superior performance beyond Llama-3-8B and long context length up to 128K context.

[27/10/23] We release ChatGLM3-6B, with the most powerful base mode under 10 billion parameters, and support for tool using, code interpreter, and agent tasks.

[25/06/23] We release ChatGLM2-6B, with stronger performance, longer context length, more efficient inference and more open license.

[14/03/23] We release ChatGLM-6B, the most popular open bilingual chatbot based on pretrained language model.

Publications

Zhengxiao Du, Aohan Zeng, Yuxiao Dong, Jie Tang (2024). Understanding Emergent Abilities of Language Models from the Loss Perspective. NeurIPS 2024.

Preprint

Aohan Zeng, Xiao Liu, Zhengxiao Du, Zihan Wang, Hanyu Lai, Ming Ding, Zhuoyi Yang, Yifan Xu, Wendi Zheng, Xiao Xia, Weng Lam Tam, Zixuan Ma, Yufei Xue, Jidong Zhai, Wenguang Chen, Peng Zhang, Yuxiao Dong, Jie Tang (2022). GLM-130B: An Open Bilingual Pre-Trained Model. ICLR 2023.

Code Preprint

Zhengxiao Du, Yujie Qian, Xiao Liu, Ming Ding, Jiezhong Qiu, Zhilin Yang, Jie Tang (2022). GLM: General Language Model Pretraining with Autoregressive Blank Infilling. the 60th Annual Meeting of the Association for Computational Linguistics, ACL 2022.

Code Preprint Paper

Xiao Liu, Kaixuan Ji, Yicheng Fu, Weng Lam Tam, Zhengxiao Du, Zhilin Yang, Jie Tang (2022). P-Tuning: Prompt Tuning Can Be Comparable to Fine-tuning Across Scales and Tasks. ACL 2022.

Code Preprint Paper

Zhengxiao Du, Chang Zhou, Jiangchao Yao, Teng Tu, Letian Cheng, Hongxia Yang, Jingren Zhou, Jie Tang (2021). CogKR: Cognitive Graph for Multi-hop Knowledge Reasoning. IEEE Transactions on Knowledge and Data Engineering, TKDE.

Code Preprint Paper

See all publications

Experience

 
 
 
 
 
Research Intern
DAMO Academy, Alibaba Group
Jun 2020 – Present Beijing
Advised by Hongxia Yang
 
 
 
 
 
Research Intern
Cornell University
Jun 2019 – Oct 2019 Ithaca, NY
Advised by Professor Thorsten Joachims
 
 
 
 
 
Research Assistant
Knowledge Engineering Group, Tsinghua University
Jun 2017 – Jun 2019 Beijing
Advised by Professor Jie Tang

Honors

Tsinghua Excellent (Bachelor) Graduate
Tsinghua University Jun 2020
Top 2% of all the graduates
The Cai Xiong Scholarship
Tsinghua University Oct 2019
Top 1%, awarded for outstanding research experience
Elite Collegiate Award
China Computer Federation Oct 2019
73 winners nationwide, only 4 in Tsinghua University

Contact