<link rel="stylesheet" href="styles.cf390c6bfbe7555f.css">
dc.contributor.authorShafique, Muhammad Ali
dc.date.accessioned2026-11-09T15:46:44Z
dc.date.graduationmonthDecember
dc.date.issued2026
dc.description.abstractThe increasing computational demands of artificial intelligence (AI) make efficient GPU execution important for training phase, inference stage, and infrastructure design. Reduced numerical precision, multi-GPU training strategies, and newer accelerators can improve performance, but their benefits depend on workload characteristics and type. Fast execution and high GPU utilization do not necessarily result in low energy consumption. This dissertation studies these trade-offs with three studies at the model, training, and infrastructure levels. At the model level, TensorFlow automatic mixed precision and TensorFlow-TensorRT FP32, FP16, and INT8 execution are evaluated using three vision architectures on CIFAR-10 and a Cats vs Dogs subset. The results show that the accuracy, memory, training time, and inference performance depend on the model architecture, precision, and batch size. At the training-system level, full-parameter fine-tuning of 3B and 8B Llama models is evaluated on eight NVIDIA H100 GPU using multi-GPU techniques such as DP, DDP, FSDP, and ZeRO. For the 3B workload, ZeRO-2 achieves approximately 6,844 tokens/kJ compared to 4,537 tokens / kJ for DDP, improving energy efficiency by 51%. More aggressive sharding does not consistently improve efficiency, and the selection of strategies depends on the size and memory requirements of the model. At the infrastructure level, eight-GPU H100 and B200 systems are compared across five language models and three vision-language models. B200 reduces the duration of language model training by almost 11-15% while consuming more GPU energy. Vision-language results reveal workload dependent platform behavior. Facility scenarios project the electricity consumption and cost implications of these differences. Together, these findings support workload aware selection of reduced precision, multi-GPU training strategies, and the newer GPU platforms under the accuracy, memory, work completion time, and energy profiles in training and inference stages.
dc.description.advisorArslan Munir
dc.description.advisorDon M. Gruenbacher
dc.description.degreeDoctor of Philosophy
dc.description.departmentDepartment of Electrical and Computer Engineering
dc.description.levelDoctoral
dc.identifier.urihttps://hdl.handle.net/2097/47497
dc.language.isoen_US
dc.subjectHigh-Performance Computing
dc.subjectLarge Language Models
dc.subjectDistributed Training
dc.subjectQuantization
dc.subjectEnergy-Efficient AI Systems
dc.subjectGPU Performance Characterization
dc.titlePerformance characterization and energy efficiency of GPU-accelerated AI systems: from deep learning optimization to distributed LLM training
dc.typeDissertation

Files

Original bundle

Now showing 1 - 1 of 1
Loading...
Thumbnail Image
Name:
MuhammadAliShafique2026.pdf
Size:
10.13 MB
Format:
Adobe Portable Document Format

License bundle

Now showing 1 - 1 of 1
Loading...
Thumbnail Image
Name:
license.txt
Size:
1.65 KB
Format:
Item-specific license agreed upon to submission
Description: