PhD Scholarship: Development of Energy Efficient Machine Learning Models

Overview

This project is a part of a 4-year dual PhD programme between National Tsing Hua University (NTHU) in Taiwan and the University of Liverpool. As part of the NTHU-UoL Dual PhD Award, students are in the unique position of being able to gain 2 PhD awards at the end of their degree from two internationally recognised world leading Universities, as well as benefiting from a rich cultural experience. Students can draw on large scale national facilities of both countries and create a worldwide network of contacts across 2 continents. It is planned that students will spend 2 years at NTHU, followed by 2 years at the University of Liverpool.

Both the University of Liverpool and NTHU have agreed to waive the tuition fees (worth £25,000 per year for international students at University of Liverpool) for the duration of the project and stipend of TWD11,000/month will be provided as a contribution to living costs (the equivalent of £280 per month when in Liverpool). Additional funding could be available for excellent candidates.

Project Description

Energy consumption has been a crucial concern due to the changes in global climate conditions during the past decades, and efficient usage of energy has attracted the attention of many researchers. However, since the prevalent usage of deep neural networks (DNNs), the need of energy consumption for training DNNs, especially for large-scale models (e.g., ResNet, Transformer, GPT-3, etc.), have surged, posing significant challenges to global energy and hence the climate conditions. These DNN models have been widely employed in various application domains such as computer vision (CV), self-driving cars, autonomous drones, surveillance cameras, natural language processing (NLP), and so on. Executing these DNNs typically requires the usage of many graphic processing units (GPUs) or DNN accelerators. As the sizes of those DNNs increase, the computational cost and the required inference power also grow.

Preliminaries studies show that light-weight models can be achieved during the testing phase [1][2], however, the training phase of big DNN remains computationally expensive. As a result, in this proposal, we plan to develop efficient DNN models, especially for vision and robotic applications (which usually require large-size DNN models), such that their inference and the training costs can be reduced. Specifically, the target research and application domains considered in this proposal include DNN models for CV, federated learning, and / or control models based on reinforcement learning (RL).

[1] Tran, Minh Q., Tuong Do, Huy Tran, Erman Tjiputra, Quang D. Tran, and Anh Nguyen. “Light-weight deformable registration using adversarial learning with distilling knowledge.” IEEE Transactions on Medical Imaging (2022).

[2] Chang, Chin-Jui, Yu-Wei Chu, Chao-Hsien Ting, Hao-Kang Liu, Zhang-Wei Hong, and Chun-Yi Lee. “Reducing the Deployment-Time Inference Control Costs of Deep Reinforcement Learning Agents via an Asymmetric Architecture.” In IEEE International Conference on Robotics and Automation (ICRA), 2021.

Deadline

As soon as possible. The candidates will be evaluated on rolling basis.

How to Apply

The students are encouraged to contact Dr. Anh Nguyen (anh.nguyen@liverpool.ac.uk) and Dr. Chun-Yi Lee (cylee@cs.nthu.edu.tw) with the CV and research proposal. Please use the subject NTHU-UoL PhD Application in your email.

The official apply link is here.