Sign-In
Cart
Building and Customizing Inference Engines for LLMs: From First Principles to Production - A Complete Guide to Designing High-Performance, Efficient,
Paperback
Publisher: Independently Published
Published: Jul 4 2026
Pages: 434
Weight: 1.65
Height: 0.88 Width: 7.00 Depth: 10.00
Language: English
Everyone is learning to use large language models. Almost no one understands how they actually run. This book closes that gap - carrying you, without a single hand-wave, from what is a token? to a working, batched, quantized, multi-GPU inference server you built yourself.
An LLM's weights hold the intelligence, but they do nothing on their own. The inference engine is the software that decides how those billions of numbers are moved, cached, batched, and multiplied against a stream of concurrent users. Get it wrong and a state-of-the-art model crawls and falls over at ten users. Get it right and the same weights on the same GPU serve dozens - at a fraction of the latency and cost.
What you will learn:
- Why decoding is memory-bandwidth-bound - the one law that explains PagedAttention, quantization, and FlashAttention as inevitable consequences, not tricks.
- The full anatomy of an engine: tokenizer, scheduler & continuous batching, memory manager, KV cache & PagedAttention, quantization (INT8/INT4/FP8), sampler, and Mixture-of-Experts routing.
- The hot path: fused GPU kernels and FlashAttention, writing your own kernels in Triton/CUDA/Rust, speculative decoding, and multi-GPU tensor/pipeline/expert parallelism.
- Real engines dissected: vLLM, TensorRT-LLM, llama.cpp, MLC-LLM, and SGLang - and how to deploy Llama, Qwen, and DeepSeek on each.
- Production skills: benchmarking, profiling, cost/SLO capacity planning, and eighteen-plus real-world war stories.
Built for three altitudes. Beginners get the concept and the mental model; intermediate engineers get the algorithms, the math, and runnable Python; advanced engineers get kernel-level detail, C++/Rust considerations, failure modes, and the frontier.
Who it's for: LLM & inference engineers, AI platform engineers and architects, forward-deployed engineers, researchers, and technology leaders who must reason about latency, throughput, memory, and cost - not just call an API.
You will build DwarfStar, a minimal but real inference engine, growing it stage by stage into a batched, quantized, C++/CUDA-accelerated server. Close this book and you will open the source of vLLM or TensorRT-LLM and feel at home.
Also in
Applications
R for Data Science: Import, Tidy, Transform, Visualize, and Model Data
Wickham, Hadley
Grolemund, Garrett
Paperback
Designing Machine Learning Systems: An Iterative Process for Production-Ready Applications
Huyen, Chip
Paperback
Microsoft 365 Excel All-In-One for Dummies
Alexander, Michael
Kusleika, Dick
Ringstrom, David H.
Paperback
R for Data Science: Import, Tidy, Transform, Visualize, and Model Data
Cetinkaya-Rundel, Mine
Wickham, Hadley
Grolemund, Garrett
Paperback
Cloud Finops: Collaborative, Real-Time Cloud Value Decision Making
Fuller, Mike
Storment, J. R.
Paperback
Microsoft Copilot Studio Step by Step: Customizing Copilot and Creating Agents
Crosbie, Lisa
Paperback
Microsoft Excel Data Analysis and Business Modeling (Office 2021 and Microsoft 365)
Winston, Wayne
Paperback
The Insider You Built: How Organizations Stay in Control of Autonomous AI Agents
Stewart Gloster, Camille
Paperback
Ingeniería de Ia (Spanish Edition): Creación de Aplicaciones Con Modelos Fundacionales
Huyen, Chip
Paperback
Excel Formulas: Quickstudy Laminated Study Guide (Quickstudy Computer)
Jensen, Krista
Hales, John
Other
Learning Generative AI Tools for Excel: Speed Up Your Everyday Tasks with Microsoft Excel, Copilot, Chatgpt, and Beyond
Duca, Angelica Lo
Paperback
An Introduction to Statistical Learning: With Applications in R
Hastie, Trevor
James, Gareth
Witten, Daniela
Paperback
Learning Tableau 2025 - Sixth Edition: Leverage Tableau's newest features to revolutionize your data storytelling with AI-enhanced insights
Milligan, Joshua N.
Paperback
The AI Workweek: Cut Hours, Automate Busywork, and Focus on What Actually Matters
Gross, Ashley
Paperback
Social Science Research Design and Statistics: A Practitioner's Guide to Research Methods and IBM SPSS Analysis
Rovai, Alfred P.
Baker, Jason D.
Ponton, Michael K.
Paperback
Planning and Control Using Oracle Primavera P6 Versions 8 to 21 PPM Professional
Harris, Paul E.
Paperback
Materials Management with SAP S/4hana: Business Processes and Configuration
Akhtar, Jawad
Murray, Martin
Hardcover
Microsoft Power Bi Data Analyst Associate Study Guide: Prepare for the Pl-300 Exam and Use Power Bi Successfully
Turley, Paul
Paperback
Planning and Control Using Oracle Primavera P6 Versions 18 to 24 PPM Professional
Harris, Paul E.
Hanak, Jan
Paperback
Planning and Control Using Oracle Primavera P6 Versions 18 to 23 PPM Professional
Harris, Paul E.
Paperback
The AI Agent Blueprint: A Practical Playbook for Building Agentic Artificial Intelligence: Launch Your First Agent in 30 Days
Daniels, Alexander J.
Paperback
Mastering QuickBooks 2026 - Seventh Edition: The complete guide to small business accounting with US QuickBooks Online
Shelton, Crystalynn
Paperback
Information Architecture: For the Web and Beyond
Arango, Jorge
Rosenfeld, Louis
Morville, Peter
Paperback
Soccer Analytics with Machine Learning: Learning Predictive Modeling Techniques with Sports Data
Gao, Haipeng
Shen, Weining
Joury, Ari
Paperback
Python in Excel: Unlocking Powerful Data Analysis and Automation Solutions
Bastick, Liam
Newitt, Kathryn
Paperback
Software Profit Streams(TM): A Guide to Designing a Sustainably Profitable Business
Tanner, Jason
Hohmann, Luke
Paperback
Data Visualization with Microsoft Power Bi: How to Design Savvy Dashboards
Kolokolov, Alex
Zelensky, Maxim
Paperback
Illustrated Microsoft(r) 365(r) & Office(r) Introductory, First Edition
Duffy, Jennifer
Wermers, Lynn
Beskeen, David
Paperback
Building Event-Driven Microservices: Leveraging Organizational Data at Scale
Bellemare, Adam
Paperback
Master Your Data with Power Query in Excel and Power Bi: Leveraging Power Query to Get & Transform Your Task Flow
Escobar, Miguel
Puls, Ken
Paperback
Reimagining Collaboration: Slack, Microsoft Teams, Zoom, and the Post-COVID World of Work
Simon, Phil
Paperback
The SaaS Email Marketing Playbook: Convert Leads, Increase Customer Retention, and Close More Recurring Revenue With Email
Garbugli, Étienne
Paperback
Regression Modeling Strategies: With Applications to Linear Models, Logistic and Ordinal Regression, and Survival Analysis
Harrell Jr, Frank E.
Paperback
Python Machine Learning By Example - Fourth Edition: Unlock machine learning best practices with real-world use cases
Liu, Yuxi (Hayden)
Paperback
AI for Gig Workers: Essential Strategies to Transform Your Freelance Career
Cerdeira, Pamela
Savage, Saiph
Savage, Liliana
Paperback
Microsoft 365 Copilot Adoption: A Practical Guide for Business Leaders and Consultants
Smith, Mark
Smith, Megan
Paperback
Managing AI Projects: Drive Innovation and Successfully Navigate the Full AI Project Lifecycle
Sánchez, Adrián González
Runtasewee, Malini Jain
Paperback
Microsoft Copilot 365 User Guide: The Complete AI Productivity Companion for Word, Excel, Outlook & Teams
Jk Stratton, Evan
Paperback
Essential Math for AI: Next-Level Mathematics for Efficient and Successful AI Systems
Nelson, Hala
Paperback
AI Can Make You Smarter: Practical Skills. Sharper Thinking. Business Value.
Kreitzberg, Charles B.
Paperback
Power BI for Beginners: The 7-Day Method to Build Dashboards and Become a Data Strategist (Video Course Included)
Pierce, Callum
Pierce, Kora
Paperback
Databricks Certified Data Engineer Associate Study Guide: In-Depth Guidance and Practice
Alhussein, Derar
Paperback
Between You and AI: Unlock the Power of Human Skills to Thrive in an Ai-Driven World
Iorio, Andrea
Hardcover
Warehouse Management with SAP S/4hana: Embedded and Decentralized Ewm
Sachan, Namita
Jain, Aman
Hardcover
Microsoft Power Bi Visual Calculations: Simplifying Dax
Ter Heerdt, Jeroen
Stikkelorum, Madzy
Lelijveld, Marc
Paperback
Me, My Customer, and AI: The New Rules of Entrepreneurship
Werdelin, Henrik
Thorne, Nicholas
Hardcover
Microsoft Excel in Easy Steps: Illustrated Using Excel in Microsoft 365
Price, Michael
McGrath, Mike
Paperback
Salesforce Certified Platform Administrator Study Guide: Launch and Elevate Your Salesforce Career
Wheeler, Mike
Paperback
ML and Generative AI in the Data Lakehouse: Building and Deploying AI Applications at Scale
Haelen, Bennie
Paperback
Dark Web: A Complete Guide to the Dark Web True Tales (The Addictive Input down able Gripping Crime Thriller of Mystery and Suspense)
Guerrero, Theodore
Paperback
Applied AI for Enterprise Java Development: Leveraging Generative Ai, Llms, and Machine Learning in the Java Enterprise
Bueno, Alex Soto
Eisele, Markus
Vinto, Natale
Paperback
Planning and Control Using Microsoft Project 365 and 2021: Including 2019, 2016 and 2013
Harris, Paul E.
Paperback
How to Measure Anything in Project Management
Budzier, Alexander
Hubbard, Douglas W.
Bang Leed, Andreas
Hardcover
High Performance Browser Networking: What Every Web Developer Should Know about Networking and Web Performance
Grigorik, Ilya
Paperback
The Developer's Playbook for Large Language Model Security: Building Secure AI Applications
Wilson, Steve
Paperback
Hexagonal Architecture Explained: How the Ports & Adapters architecture simplifies your life, and how to implement it, Updated 1st Ed.
Cockburn, Alistair
Garrido de Paz, Juan Manuel
Paperback
How AI Ate the World: A Brief History of Artificial Intelligence - And Its Long Future
Stokel-Walker, Chris
Paperback
Aprenda Visual Basic (Vba) Y Macros Para Excel: Más de 100 ejercicios resueltos, macros y juegos, para desarrollar tus habilidades de programación
Vidal Bosch, Josep Ramon
Paperback
Boost Your Skills In Computer Basics for ESL Learners: (+ Online Simulations & Resources)
Wooden, Lois
Paperback
Modern Data Analytics in Excel: Using Power Query, Power Pivot, and More for Enhanced Data Analytics
Mount, George
Paperback
Beyond Bullet Points: Using PowerPoint to Tell a Compelling Story That Gets Results
Atkinson, Cliff
Paperback
Quickbooks Pro Desktop User Guide 2024: A Step-by-Step Handbook for Efficient Bookkeeping and Finance Management
Duncan, Michel M.
Paperback
Articulate Storyline 360: The Essentials and Beyond [2026 Edition]: A Hands-On, Project-Based Classroom in a Book for Building Engaging eLearning
Hadi, Kal
Siegel, Kevin
Paperback
Numerical Python: Scientific Computing and Data Science Applications with Numpy, Scipy and Matplotlib
Johansson, Robert
Paperback
Generative AI for Software Development: Building Software Faster and More Effectively
Pereira, Sergio
Paperback
Simply Learn QuickBooks Online: The Complete Illustrated Step-by-Step Guide for Beginners
Brownell, Joseph
Paperback
Microsoft Excel VBA and Macros: Your Guide to Efficient Automation
Jelen, Bill
Syrstad, Tracy
Paperback
Deep Learning for Biology: Harness AI to Solve Real-World Biology Problems
Ravarani, Charles
Latysheva, Natasha
Paperback
SAP Cloud Integration Cookbook: Advanced Cloud Integration with SAP Integration Suite
Koch, Martin
Reisinger, Thorsten
Urschick, Marc
Hardcover
Machine Learning Design Patterns: Solutions to Common Challenges in Data Preparation, Model Building, and MLOps
Munn, Michael
Lakshmanan, Valliappa
Robinson, Sara
Paperback
Learn Excel 2019 for Mac Expert Skills with The Smart Method: Tutorial teaching Advanced Techniques
Smart, Mike
Paperback
The Big Book of Dashboards: Visualizing Your Data Using Real-World Business Scenarios
Cotgreave, Andy
Wexler, Steve
Shaffer, Jeffrey
Paperback
Python Machine Learning - Second Edition: Machine Learning and Deep Learning with Python, scikit-learn, and TensorFlow
Raschka, Sebastian
Mirjalili, Vahid
Paperback