Recent breakthroughs in Natural Language Processing (NLP) demonstrate the ability of Large Language Models - LLM (such as GPT-3 and T5) to solve diverse problems. However, building such large models raises challenges on data, training, and deployment considerations. Recently, the Big Science collaborative initiative - a group of over 1,000 researchers from academia and industry - developed BLOOM, an open source massive 176 billion parameter multilingual model.

In this webinar, we will present the key learnings of the BigScience initiative in developing the BLOOM language model in a transparent and collaborative way. This is an opportunity to discover what motivated the creation of the workshop, and the 4 month training process on the Jean Zay (IDRIS) supercomputer* using 384 NVIDIA A100 GPUs, and discuss the BLOOM achievements.

• why large language models matter

• an overview of the BigScience project

• how to build multilingual large language model (dataset collection, engineering challenges, evaluation tools)

• discuss LLM deployment considerations

Content

content goes here