70016

Natural Language Processing

Module aims

In this module you will have the opportunity to:
- Learn the foundations, key concepts, and real-world applications of Natural Language Processing (NLP), with an emphasis on deep learning approaches.
- Study models for representing words and their meanings.
- Use these representations in text classification tasks like Natural Language Inference (NLI).
- Analyse deep learning model architectures used in Language modeling and text generation.
- Review current methods for training and fine-tuning Large Language Models (LLMs).   

Learning outcomes

Upon successful completion of this module you will be able to:
- describe and critically appraise modern NLP approaches and techniques.
- model textual data using appropriate language representations.
- discuss various NLP approaches to classification tasks (e.g. NLI).
- apply state-of-the art tools and techniques to solve real-world NLP problems.
- analyse deep learning based model architectures used in language modeling and text generation
- explain the process of training and fine-tuning of LLMs.
- compare the performance and outputs of various LLMs.

Module syllabus

Introduction to NLP  
Word meaning and representations
Classification tasks
Language models (n-gram based, RNN, Attention-based, Transformer, GPT)
Large Language Models
Benchmarking and Evaluation
Advanced NLP Topics

Teaching methods

The material will primarily be delivered through lectures, complemented by weekly lab sessions. These labs feature unassessed formative tutorial exercises designed to deepen your understanding of lecture content and support your preparation for both the coursework and the final exam. Specimen answers will be provided for these exercises, and the sessions will be facilitated by Graduate Teaching Assistants (GTAs). Where possible, guest lecturers will be invited to offer additional perspectives and introduce advanced topics. An online discussion forum will be used to support the module and foster open communication and collaboration among students.

Assessments

As mentioned above, you will be provided with formative tutorial exercises, along with specimen answers and GTA support, in the lab sessions which are designed to help you prepare for the coursework and the exam. There will be one coursework that contributes 30% of the mark for the module. The coursework will be a real NLP shared task from popular NLP conferences and workshops. It will be designed to give you the experience of NLP research. There will be a final written exam, which accounts for the remaining 70% of the marks, covering the content taught in the lectures.
                
In the lab sessions which consist of formative tutorial exercises, GTAs will help you and provide verbal feedback. You will also be provided with a detailed written feedback on the coursework.

Module leaders

Dr Marek Rei
Dr Chiraag Lala