Offered by Department of Mechanical Engineering, IIT Bombay

This module begins with the foundations of AI and machine learning, hardware-aware programming, and the fundamentals of parallel computing. Before moving to distributed systems, an AI Engineer must first understand the efficiency of individual compute nodes. You will learn how different hardware architectures, such as central and graphical processing units, impact the performance of AI training loops. By understanding memory hierarchies and cache optimisation, you will learn to identify and resolve bottlenecks in standard machine learning workloads. The module then covers shared memory parallelisation, teaching you to utilise multi-core processors effectively for matrix operations and feature engineering. Further, the module introduces Message Passing Interface (MPI) for distributed ML systems and addresses a key element of AI Engineering: enabling multiple machines to function as a single, cohesive training unit using advanced communication patterns and data parallelism.

This module shifts focus to orchestrating multi-node clusters and managing containerisation, once you understand single-node performance and the Message Passing Interface (MPI). You will explore how containerisation ensures reproducibility across diverse environments and how orchestration tools manage these containers at scale. This module covers resource management and job scheduling, essential for navigating shared high-performance computing clusters. Furthermore, you will also learn how to leverage distributed deep learning frameworks to run ML workloads across a network of multiple nodes using MPI and data parallelism.

This module is at the core of AI Engineering and explores the specific infrastructure requirements for large-scale model training, such as Large Language Models (LLMs). You will learn various model parallelism strategies, including tensor and pipeline parallelism, to handle models that exceed the memory of a single GPU. To support the high-speed parallelised models, it is critical to have data engineering systems operating at scale. Hence, a significant portion of this module is dedicated to building high-performance data pipelines—understanding how parallel file systems and high-performance I/O libraries prevent data starvation during training. In addition to model parallelism and data engineering, you will also develop proficiency in profiling distributed systems to detect load imbalances and communication overhead, ensuring that your AI Engineering solutions are truly optimised for performance. Finally, this module explores ML model serving and inference frameworks to take large-scale models from training environment to end users.

This module focuses on the "Operations" of AI, moving from model training to production-grade deployment. You will learn to build continuous integration and delivery (CI/CD) pipelines tailored for AI, incorporating automated testing for both code and data, deployment and experiment tracking. Further, the module covers production monitoring and observability, teaching you how to track model performance and detect data drift in real-time. Finally, you will explore cloud-HPC integration, learning how to provision hybrid infrastructures that combine on-premise clusters with third-party cloud capabilities to ensure your MLOps strategy is flexible. The module ends with an overview of advanced topics and emerging AI Engineering trends.