3

Scaling Speech-Text Pre-training with Synthetic Interleaved Data
peech language models (SpeechLMs) accept speech input and produce speech output, allowing for more natural human-computer interaction …
Preprint
Understanding Emergent Abilities of Language Models from the Loss Perspective
Recent studies have put into question the belief that emergent abilities in language models are exclusive to large models. This …
Preprint
GLM-130B: An Open Bilingual Pre-Trained Model
We introduce GLM-130B, a bilingual (English and Chinese) pre-trained language model with 130 billion parameters. It is an attempt to …
Code Preprint
GPT Understands, Too
While GPTs with traditional fine-tuning fail to achieve strong results on natural language understanding (NLU), we show that GPTs can be …
Preprint Code