PhD Studentship: Sustainable and fast inference and fine-tuning for large language models

PhD Supervisor: Dr. Zhanxing Zhu

Supervisory Staff: Dr. Zhanxing Zhu and Dr. Stuart Middleton

Venture description:

Just lately, giant language fashions (LLMs), equivalent to OpenAI’s ChatGPT and Google’s Bard, have caught monumental consideration of the general public. They will generate remarkably life like, coherent textual content based mostly on a consumer’s enter and have the potential to be general-purpose instruments used all through society, e.g. for customer support, summarizing texts, answering questions, code technology and even fixing math issues. These LLMs are usually with Transformer architectures with tens of billions of parameters and skilled with trillions of tokens.

Nonetheless, the big mannequin measurement and excessive computational complexity of LLMs ends in vital computational energy and storage necessities far surpassing what’s presently out there with customary client {hardware}. As an example, even the comparatively modest-sized mannequin LLaMA-65B wants 130GB of GPU RAM for inference and greater than 780 GB for fine-tuning with new information.  Due to this fact, the intention of this Ph.D venture is to develop quick and environment friendly inference and fine-tuning methods for LLMs such that they are often accessible and deployed given restricted computational sources. This may promote sustainable language mannequin utilization in additional eventualities by way of energy and {hardware} consumption.  

On this venture, the scholar will discover varied pathways to attain sustainable LLM inference and tuning, together with:

• Mannequin quantization, i.e. quantizing high-bit values into low-bit ones like 4-bit, together with weight quantization, KV cache quantization from an information-theoretical perspective. Notably, job tailored quantization, the place quantization technique may think about info from the anticipated goal job corpus distribution in addition to authentic LLM pre-training corpus.
• Knowledge choice throughout fine-tuning, the place choice mechanisms of knowledge might be developed as an alternative of utilizing complete coaching dataset;
• Mannequin compression and distillation, i.e. compressing giant fashions into tiny ones with considerably smaller variety of parameters.

College students will work in a collaborative crew with each consultants of machine studying and pure language processing concerned. We strongly counsel that college students with sturdy motivation dive into the analysis of sustainable giant language fashions to make GPT-type fashions accessible even with restricted computational sources.  

It is a 4-year built-in PhD (iPhD) programme and is a part of the UKRI AI Centre for Doctoral Coaching in AI for Sustainability (SustAI). For extra details about SustAI, please see: https://sustai.information/

In the event you want to talk about any particulars of the venture informally, please contact Professor Enrico Gerding, Director of the SustAI CDT, E-mail: [email protected] .

Entry Necessities

An excellent undergraduate diploma (at the least a UK 2:1 honours diploma, or its worldwide equal).

Cut-off date : 8 April 2024.

Purposes might be thought-about within the order that they’re obtained.

Funding: We provide a variety of funding alternatives for each UK and worldwide college students, together with Bursaries and Scholarships.  For extra info please go to PhD Scholarships Doctoral School College of Southampton   Funding might be awarded on a rolling foundation, so apply early for the most effective alternative to be thought-about.

How To Apply

Apply on-line, by clicking the ‘Apply’ button, above.

Choose programme kind (Analysis), 2024/25, School of Engineering and Bodily Sciences, subsequent web page choose “PhD iPhD AI for Sustainability (Full time)”. In Part 2 of the applying kind it is best to insert the title of the supervisor

Purposes ought to embody:

• Analysis Proposal
• Curriculum Vitae
• Two reference letters
• Diploma Transcripts/Certificates so far

For additional info please contact: [email protected]