Machine Learning Systems

Principles and Practices of Engineering Artificially Intelligent Systems

Author, Editor & Curator
Affiliation

Harvard University

Last Updated

January 17, 2025

Abstract

Machine Learning Systems presents a comprehensive approach to understanding and engineering machine learning (ML). While many resources focus on ML algorithms and model architectures, this book serves as a bridge between theoretical foundations and practical engineering. It emphasizes the systems context that engineers need to master when building AI solutions in the real world. The text progresses from foundational concepts to advanced system design, integrating topics such as data engineering, model optimization, hardware-aware training approaches, and inference acceleration strategies. Throughout the book, readers develop a principled understanding of ML systems engineering, learning to reason about system architectures and address critical challenges in areas including security, privacy, and reliability. While ML applications and tools evolve rapidly, the engineering principles for building ML systems remain largely consistent. This book distills these enduring concepts, making it a resource for anyone seeking to build flexible, efficient, and robust ML systems.


🎙 Listen to the AI Podcast, created using Google’s Notebook LM and inspired by insights drawn from our IEEE education viewpoint paper. This podcast provides an accessible overview of what this book is all about.

 


Cover image.

Preface

Welcome to Machine Learning Systems, your gateway to the fast-paced world of machine learning (ML) systems. This book is an extension of the CS249r course at Harvard University, taught by Prof. Vijay Janapa Reddi, and is the result of a collaborative effort involving students, professionals, and the broader community of AI practitioners.

We’ve created this open-source book to demystify the process of building efficient and scalable ML systems. Our goal is to provide a comprehensive guide that covers the principles, practices, and challenges of developing robust ML pipelines for deployment. This isn’t a static textbook—it’s a living, evolving resource designed to keep pace with advancements in the field.

“If you want to go fast, go alone. If you want to go far, go together.” – African Proverb

As a living and breathing resource, this book is a continual work in progress, reflecting the ever-evolving nature of machine learning systems. Advancements in the ML landscape drive our commitment to keeping this resource updated with the latest insights, techniques, and best practices. We warmly invite you to join us on this journey by contributing your expertise, feedback, and ideas.

Global Outreach

Thank you to all our readers and visitors. Your engagement with the material keeps us motivated.

Why We Wrote This Book

While there are plenty of resources that focus on the algorithmic side of machine learning, resources on the systems side of things are few and far between. This gap inspired us to create this book—a resource dedicated to the principles and practices of building efficient and scalable ML systems.

Our vision for this book and its broader mission is deeply rooted in the transformative potential of AI and the need to make AI education globally accessible to all. To learn more about the inspiration behind this project and the values driving its creation, we encourage you to read the Author’s Note.

Want to Help Out?

This is a collaborative project, and your input matters! If you’d like to contribute, check out our contribution guidelines. Feedback, corrections, and new ideas are welcome—simply file a GitHub issue.

What’s Next?

If you’re ready to dive deeper into the book’s structure, learning objectives, and practical use, visit the About the Book section for more details.