PyTorch Systems Engineering
Hardcover
ISBN13: 9786048654283
Publisher: Devlin Nexley
Published: Sep 16 2026
Pages: 300
Weight: 1.25
Height: 0.69 Width: 6.00 Depth: 9.00
Language: English
PyTorch Systems Engineering is a deep, architecture-driven exploration of PyTorch as a modern execution platform for large-scale AI systems. It moves beyond model building and API usage to focus on the internal mechanisms that define performance, scalability, and reliability in production deep learning workloads.
As AI systems grow in complexity-from large language models to distributed multimodal pipelines-the challenges shift away from model design and toward systems engineering: how tensors are executed across heterogeneous hardware, how autograd constructs and traverses dynamic computation graphs, how GPU memory is allocated and optimized under pressure, and how distributed training systems maintain correctness and efficiency at scale.
This book treats PyTorch not as a library, but as a full runtime system composed of tightly integrated subsystems: a dynamic execution engine, a tensor computation backend, a compiler pipeline, a distributed coordination layer, and a GPU-accelerated memory management system.
Throughout the book, readers will gain a practical understanding of how PyTorch actually behaves under production workloads, including its performance characteristics, internal abstractions, and failure modes.
You will learn how to reason about deep learning systems in terms of execution flow, memory behavior, and infrastructure constraints rather than isolated model architectures.
Key topics include:
PyTorch runtime architecture and execution model
Tensor memory layout, storage semantics, and GPU data movement
Autograd internals and dynamic computation graph construction
CUDA execution, kernel optimization, and GPU performance tuning
Mixed precision training and numerical stability in large models
Compiler pipelines including graph capture and kernel fusion systems
Distributed training architectures such as DDP and FSDP
Transformer and large language model training systems
Memory optimization strategies for large-scale workloads
Production inference pipelines and deployment architectures
PyTorch extensibility through C++ extensions and CUDA kernels
Debugging distributed systems and runtime failure modes
By the end of this book, readers will be able to design, optimize, and operate production-grade AI systems built on PyTorch, with a clear understanding of the tradeoffs that govern performance, scalability, and system reliability.
This is not an introductory guide to machine learning. It is a systems engineering manual for building and scaling modern AI infrastructure with PyTorch.
Also from
Nexley, Devlin
Also in
Programming
Interactive Theorem Proving and Program Development: Coq'art: The Calculus of Inductive Constructions
Castéran, Pierre
Bertot, Yves
Hardcover
The Legend of Zelda(tm) Tears of the Kingdom - The Complete Official Guide: Collector's Edition
Piggyback
Hardcover
The Legend of Zelda(tm) Tears of the Kingdom - The Complete Official Guide: Standard Edition
Piggyback
Paperback
Python Crash Course, 3rd Edition: A Hands-On, Project-Based Introduction to Programming
Matthes, Eric
Paperback
Crisis Engineering: Time-Tested Tools for Turning Chaos Into Clarity
Dickerson, Mikey
Nitze, Marina
Weaver, Matthew
Paperback
Accelerate: The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations
Forsgren Phd, Nicole
Humble, Jez
Kim, Gene
Paperback
Evals for AI Engineers: Systematically Measuring and Improving AI Applications
Shankar, Shreya
Husain, Hamel
Paperback
Embedded Systems with ARM Cortex-M Microcontrollers in Assembly Language and C: Fourth Edition
Zhu, Yifeng
Paperback
Fundamentals of Software Architecture: A Modern Engineering Approach
Richards, Mark
Ford, Neal
Paperback
Site Reliability Engineering: How Google Runs Production Systems
Leng, Christof
Beyer, Betsy
Jones, Chris
Paperback
Make: Electronics: Learning by Discovery: A Hands-On Primer for the New Electronics Enthusiast
Platt, Charles
Paperback
The Manager's Path: A Guide for Tech Leaders Navigating Growth and Change
Fournier, Camille
Paperback
Vibe Coding: Building Production-Grade Software with Genai, Chat, Agents, and Beyond
Yegge, Steve
Kim, Gene
Paperback
AI Projects with Raspberry Pi: High-Performance Artificial Intelligence for Robotics, Security, Home Automation, and Vision
Hattersley, Lucy
Jepson, Brian
Paperback
I Have an App Idea: The Essential Guide to Building an App Without Tech Skills
Spann, Amanda
Paperback
The Staff Engineer's Path: A Guide for Individual Contributors Navigating Growth and Change
Reilly, Tanya
Paperback
The Devops Handbook, 2nd Edition: How to Create World-Class Agility, Reliability, & Security in Technology Organizations
Humble, Jez
Kim, Gene
Debois, Patrick
Paperback
The Return on AI: From Promise to Profit in the Age of Intelligent Business
Sawhney, Mohanbir
Mittal, Ashwin
Paperback
The Developer's Guide to AI: A Field Guide for the Working Developer
Thompson, Danny
Orshalick, Jacob
Reghunadh, Jerry M.
Paperback
Forecasting: Principles and Practice, the Pythonic Way
Athanasopoulos, George
Hyndman, Rob J.
Paperback
Software Architecture: The Hard Parts: Modern Trade-Off Analyses for Distributed Architectures
Richards, Mark
Sadalage, Pramod
Ford, Neal
Paperback
Building Applications with AI Agents: Designing and Implementing Multiagent Systems
Albada, Michael
Paperback
Observability Engineering: Achieving Production Excellence
Fong-Jones, Liz
Miranda, George
Majors, Charity
Paperback
A Human's Guide to Machine Intelligence: How Algorithms Are Shaping Our Lives and How We Can Stay in Control
Hosanagar, Kartik
Paperback
Cracking the Coding Interview: 189 Programming Questions and Solutions
McDowell, Gayle Laakmann
Paperback
Building Safer Technology: A Field Guide to Failing Well
Coles, Neil
Kissner, Lea
Zunger, Yonatan
Paperback
Linux Basics for Hackers, 2nd Edition: Getting Started with Networking, Scripting, and Security in Kali
Occupytheweb
Paperback
Learning Web Design: A Beginner's Guide to Html, Css, Javascript, and Web Images
Robbins, Jennifer
Paperback
The Pragmatic Programmer: Your Journey to Mastery, 20th Anniversary Edition
Hunt, Andrew
Thomas, David
Hardcover
Building Resilient Distributed Systems: Patterns and Practices for Stable Software
Newman, Sam
Paperback
The New AI Cold War: Liberty vs. Tyranny in the Age of Machine Empires
Maginnis, Ltc Robert L.
Paperback
SQL QuickStart Guide: The Simplified Beginner's Guide to Managing, Analyzing, and Manipulating Data With SQL
Shields, Walter
Hardcover
Java in a Nutshell: A Desktop Quick Reference
Clark, Jason R.
Flanagan, David
Evans, Benjamin J.
Paperback
Grokking Algorithms, Second Edition: An Illustrated Guide for Programmers and Other Curious People
Bhargava, Aditya Y.
Paperback
Learning Domain-Driven Design: Aligning Software Architecture and Business Strategy
Khononov, Vlad
Paperback
Generative AI Design Patterns: Solutions to Common Challenges When Building Genai Agents and Applications
Hapke, Hannes
Lakshmanan, Valliappa
Paperback
Concrete Mathematics: A Foundation for Computer Science
Knuth, Donald
Graham, Ronald
Patashnik, Oren
Hardcover
Signals & Levers: Systems Thinking Tools to Unblock Software Delivery
Hendrickson, Elisabeth
Tosi, Joel
Paperback
Head First Design Patterns: Building Extensible and Maintainable Object-Oriented Software
Freeman, Eric
Robson, Elisabeth
Paperback
The Software Architect Elevator: Redefining the Architect's Role in the Digital Enterprise
Hohpe, Gregor
Paperback
The Official Raspberry Pi Handbook 2026: Astounding Projects with Raspberry Pi Computers
Makers of Raspberry Pi Official Magazine, The
Paperback
The Multiplier Mindset: How to Move from Senior Engineer to Tech Leader in the Age of AI
Ojstersek, Gregor
Paperback
Learning Generative AI Tools for Excel: Speed Up Your Everyday Tasks with Microsoft Excel, Copilot, Chatgpt, and Beyond
Duca, Angelica Lo
Paperback
AI Agents with MCP: Model Context Protocol for Building Clients, Servers, and End-To-End Agents
Stratis, Kyle
Paperback
Build an AI Agent (from Scratch): Agents That Reason, Plan, and ACT Autonomously
Song, Younghee
Hur, Jungjun
Paperback
The Book of Debugging: A Systematic Workflow for Finding and Fixing Bugs
Kuhlmann, Johannes
Paperback
Building AI-Powered Products: The Essential Guide to AI and Genai Product Management
Nika, Marily
Paperback
Debugging: The 9 Indispensable Rules for Finding Even the Most Elusive Software and Hardware Problems
Agans, David J.
Paperback
Living a Jewish Life, Revised and Updated: Jewish Traditions, Customs, and Values for Today's Families
Cooper, Howard
Diamant, Anita
Paperback
Software Engineering at Google: Lessons Learned from Programming Over Time
Manshreck, Tom
Wright, Hyrum
Winters, Titus
Paperback
Learning Php, MySQL & JavaScript: A Step-By-Step Guide to Creating Dynamic Websites
Nixon, Robin
Paperback
Programming Massively Parallel Processors: A Hands-On Approach
El Hajj, Izzat
Hwu, Wen-Mei W.
Kirk, David B.
Paperback
Learn Powershell in a Month of Lunches, Fourth Edition: Covers Windows, Linux, and macOS
Plunk, Travis
Petty, James
Leonhardt, Tyler
Paperback
Programming Rust: Fast, Safe Systems Development
Blandy, Jim
Orendorff, Jason
Tindall, Leonora F. S.
Paperback
Certified Associate in Project Management (Capm)(R) Exam Official Cert Guide
Thomas, Arthur
Lechler, Thomas
Kanabar, Vijay
Paperback
Frictionless: 7 Steps to Remove Barriers, Unlock Value, and Outpace Your Competition in the AI Era
Forsgren, Nicole
Noda, Abi
Paperback
Electronic Music and Sound Design - Theory and Practice with Max 9 - Volume 1 (Fifth Edition)
Maurizio, Giri
Cipriani, Alessandro
Paperback
Head First Algorithms and Data Structures: A Learner's Guide to the Foundations of Computing
Freeman, Eric
Robson, Elisabeth
Paperback
Python Programming for Young Coders: A Hands-On, Project-Based Introduction to Coding for Beginners, Kids, and Teens
Pandey, Anand
Paperback
Creación de Aplicaciones Con Agentes de Ia (Spanish Edition): Diseño E Implementación de Sistemas Multiagente
Albada, Michael
Paperback
Practical AI Security: A Hands-On Guide to Attacking, Defending, and Securing Modern AI Systems
Farlow, Harriet
Paperback
Computer Science from Scratch: Building Interpreters, Art, Emulators and ML in Python
Kopec, David
Paperback
