PhD Studentship: Sustainable and fast inference and fine-tuning for large language models

PhD Supervisor: Dr. Zhanxing Zhu

Supervisory Group: Dr. Zhanxing Zhu and Dr. Stuart Middleton

Venture description:

Not too long ago, giant language fashions (LLMs), akin to OpenAI’s ChatGPT and Google’s Bard, have caught huge consideration of the general public. They’ll generate remarkably sensible, coherent textual content based mostly on a person’s enter and have the potential to be general-purpose instruments used all through society, e.g. for customer support, summarizing texts, answering questions, code technology and even fixing math issues. These LLMs are sometimes with Transformer architectures with tens of billions of parameters and skilled with trillions of tokens.

Nonetheless, the massive mannequin measurement and excessive computational complexity of LLMs ends in important computational energy and storage necessities far surpassing what’s presently accessible with commonplace shopper {hardware}. For example, even the comparatively modest-sized mannequin LLaMA-65B wants 130GB of GPU RAM for inference and greater than 780 GB for fine-tuning with new knowledge.  Due to this fact, the purpose of this Ph.D mission is to develop quick and environment friendly inference and fine-tuning methods for LLMs such that they are often accessible and deployed given restricted computational assets. It will promote sustainable language mannequin utilization in additional eventualities when it comes to energy and {hardware} consumption.  

On this mission, the scholar will discover varied pathways to attain sustainable LLM inference and tuning, together with:

• Mannequin quantization, i.e. quantizing high-bit values into low-bit ones like 4-bit, together with weight quantization, KV cache quantization from an information-theoretical perspective. Significantly, activity tailored quantization, the place quantization methodology might think about info from the anticipated goal activity corpus distribution in addition to unique LLM pre-training corpus. 
• Information choice throughout fine-tuning, the place choice mechanisms of knowledge might be developed as a substitute of utilizing complete coaching dataset; 
• Mannequin compression and distillation, i.e. compressing giant fashions into tiny ones with considerably smaller variety of parameters.  

College students will work in a collaborative crew with each specialists of machine studying and pure language processing concerned. We strongly recommend that college students with robust motivation dive into the analysis of sustainable giant language fashions to make GPT-type fashions accessible even with restricted computational assets.  

It is a 4-year built-in PhD (iPhD) programme and is a part of the UKRI AI Centre for Doctoral Coaching in AI for Sustainability (SustAI). For extra details about SustAI, please see: https://sustai.data/

If you happen to want to focus on any particulars of the mission informally, please contact Professor Enrico Gerding, Director of the SustAI CDT, E mail: [email protected] .

Entry Necessities

An excellent undergraduate diploma (at the least a UK 2:1 honours diploma, or its worldwide equal).

Cut-off date : 8 April 2024.

Purposes might be thought-about within the order that they’re acquired.

Funding: We provide a variety of funding alternatives for each UK and worldwide college students, together with Bursaries and Scholarships.  For extra info please go to PhD Scholarships Doctoral Faculty College of Southampton   Funding might be awarded on a rolling foundation, so apply early for the very best alternative to be thought-about.

How To Apply

Apply on-line, by clicking the ‘Apply’ button, above.

Choose programme kind (Analysis), 2024/25, School of Engineering and Bodily Sciences, subsequent web page choose “PhD iPhD AI for Sustainability (Full time)”. In Part 2 of the appliance type it is best to insert the identify of the supervisor

Purposes ought to embrace:

• Analysis Proposal
• Curriculum Vitae
• Two reference letters
• Diploma Transcripts/Certificates up to now

For additional info please contact: [email protected]