Optimizing Distributed Deep Neural Network (DNN) Inference for Low Latency and Energy Efficiency on Resource-Constrained Edge Devices

This research project aims to develop and evaluate novel methods for optimizing distributed deep neural network (DNN) inference to achieve low latency and energy efficiency on resource-constrained edge devices. Current approaches for distributed DNN inference typically involve splitting neural network layers or operations across multiple edge devices. However, these configurations are often pre-determined and fail to adapt to fluctuations in network latency and bandwidth, resulting in performance degradation and increased energy consumption.

To overcome these limitations, the project will leverage fine-grained monitoring of CPU, memory, and network resources to dynamically adjust DNN partitioning in response to changing conditions. The project will introduce adaptive DNN partitioning techniques that dynamically distribute layers or operations across multiple edge devices, optimizing communication overhead and balancing the computational load based on real-time resource availability, all while minimizing reconfiguration costs. The proposed techniques will be evaluated on a heterogeneous testbed comprising Raspberry Pi and NVIDIA Jetson Nano devices.

  • Faculty: Palden Lama
  • Department: Computer Science 
  • Open Positions: 3
  • Mode: Hybrid
  • Hours per week: 19
Requirements and Responsibilities: 

Needed Skills: Prerequisite course: CS 3423 Systems Programming
Some experience with Machine Learning/Deep Learning.

Student Responsibilities: 

  • Assist in setting up and configuring the Raspberry Pi and NVIDIA Jetson Nano devices for experiments.
  • Collect and analyze performance data, including inference latency, energy consumption, and resource utilization.
  • Benchmark the performance of baseline methods to compare against the proposed techniques.

    Software Development and Testing:
  • Help implement and test the adaptive DNN partitioning algorithms under the supervision of the research team.
  • Develop scripts for automating experiments and data collection processes.
  • Conduct unit testing and debugging to ensure the reliability of the implemented methods.

    Resource Monitoring and Analysis:
  • Work on integrating fine-grained resource monitoring tools to track CPU, memory, and network usage.
  • Analyze the collected data to understand system performance and identify potential optimization opportunities.

    Documentation and Reporting:
  • Document experimental procedures, results, and insights in a clear and organized manner.
  • Prepare progress reports and contribute to research papers or presentations as needed.

    Collaboration and Meetings:
  • Participate in regular project meetings to discuss progress, share findings, and receive feedback from the research team.
  • Collaborate with graduate students and other team members, contributing to the overall success of the project.