Sign-In
Cart
Pipeline Engineer: Building Modern Data Infrastructure with Python, Airflow, dbt, and the Cloud
Paperback
Publisher: Independently Published
Published: Jun 5 2026
Pages: 288
Weight: 0.77
Height: 0.72 Width: 6.00 Depth: 9.00
Language: English
Data systems are the backbone of modern organizations.
From analytics dashboards and business intelligence to machine learning pipelines and real-time decision systems, companies depend on reliable data infrastructure to operate effectively.
Pipeline Engineer is a practical, engineering-focused guide to building modern data platforms using Python, Apache Airflow, dbt, and cloud-native infrastructure.
This book teaches developers and data engineers how to design, orchestrate, transform, monitor, and scale production-grade data systems.
Why modern data engineering matters
Organizations today face challenges such as:
- fragmented data sources
- unreliable pipelines and failed jobs
- poor data quality and governance
- scaling transformation workloads
- operational complexity across cloud systems
- maintaining observability and lineage
Building dependable data infrastructure requires both software engineering discipline and operational reliability.
What you will learn
- fundamentals of modern data architecture
- designing ETL and ELT workflows
- workflow orchestration with Airflow
- transformation modeling with dbt
- scalable data ingestion patterns
- data warehouse and lakehouse concepts
- pipeline testing and validation
- observability and monitoring strategies
- cloud-native deployment workflows
- security, governance, and access management
From raw data to reliable platforms
Throughout the book, you will learn how to:
- design maintainable data pipelines
- orchestrate complex workflow dependencies
- build reusable transformation layers
- improve data quality and reliability
- monitor pipelines proactively
- scale data infrastructure across cloud environments
- manage production operations confidently
Each chapter focuses on practical workflows used in real-world data engineering teams.
Practical applications
- analytics engineering platforms
- business intelligence pipelines
- machine learning data infrastructure
- event-driven data systems
- cloud-native ETL and ELT platforms
- enterprise reporting and governance systems
These examples reflect real production data engineering challenges.
Who this book is for
- data engineers
- analytics engineers
- backend developers
- cloud engineers
- machine learning infrastructure teams
- software engineers transitioning into data platforms
If you want to build scalable, maintainable, and production-ready data systems, this book provides the roadmap.
Move data reliably.
Transform intelligently.
Engineer infrastructure that scales.
Also from
Boozman, Richard
Programación profesional en Lua: Cómo diseñar sistemas embebidos y de juegos de alto rendimiento
Boozman, Richard
Paperback
Lua für Anfänger: Wie man einfache Skriptsprachen für Spiele, Apps und eingebettete Systeme lernt
Boozman, Richard
Paperback
The AI Builder: Create Generative AI Applications with Python, LangChain, and CuttingEdge Foundation Models
Boozman, Richard
Paperback
See, Read, Reason: Building Multimodal AI Applications That Understand Images, Text, and Audio Together
Boozman, Richard
Paperback
Agents at Work: Designing Autonomous AI Workflows, Tool-Calling Agents, and MultiAgent Systems with Python
Boozman, Richard
Paperback
Reward and Learn: Practical Reinforcement Learning for Autonomous Agents, Games, and Robot Control
Boozman, Richard
Paperback
Tiny but Mighty: Edge AI Engineering: Quantization, Pruning, and On-Device ML for Embedded Systems
Boozman, Richard
Paperback
Swift and Sure: Building Beautiful, High-Performance iOS Applications with SwiftUI and Swift Concurrency
Boozman, Richard
Paperback
iOS at Speed: Performance Engineering, Profiling, Memory Management, and Optimization for Apple Platforms
Boozman, Richard
Paperback
Kotlin Craft: Professional Android Development: Architecture, Coroutines, and Production Practices
Boozman, Richard
Paperback
Von Natur Aus: Entwicklung leistungsstarker, nativer mobiler Apps mit React Native und Expo
Boozman, Richard
Paperback
Stream Everything: Real-Time Data Processing, Event-Driven Architectures, and Kafka Engineering with Python
Boozman, Richard
Paperback
Reiss Dich Zusammen: Moderne Android-UI-Entwicklung mit Jetpack Compose, Zustandsverwaltung und Animation
Boozman, Richard
Paperback
Vom Modell Zum Markt: MLOps Engineering: Versionierung, Überwachung und Bereitstellung von KI-Modellen in der Produktion
Boozman, Richard
Paperback
Agenten Im Einsatz: Entwicklung autonomer KI-Workflows, Tool-AufrufAgenten und Multiagentensysteme mit Python
Boozman, Richard
Paperback
Pipeline-Ingenieur: Aufbau einer modernen Dateninfrastruktur mit Python, Airflow, dbt und der Cloud
Boozman, Richard
Paperback
Learn from Data: Statistical Machine Learning with Python: Regression, Classification, and Neural Nets
Boozman, Richard
Paperback
From Model to Market: MLOps Engineering: Versioning, Monitoring, and Deploying AI Models in Production
Boozman, Richard
Paperback
The Knowledge Engine: Building RAG Systems: Retrieval-Augmented Generation with Python and Vector Databases
Boozman, Richard
Paperback
Responsible by Design: AI Safety, Alignment, and Trust Engineering for Production Machine Learning Systems
Boozman, Richard
Paperback
LLM Engineer: Build, Fine-Tune, and Deploy Production-Grade AI Applications with Python and Modern LLMs
Boozman, Richard
Paperback
See Like a Machine: Computer Vision Mastery with PyTorch: Detection, Segmentation, and Real-Time Inference
Boozman, Richard
Paperback
Guía intermedia de Gleam: Cómo construir sistemas concurrentes y tolerantes a fallos con confianza
Boozman, Richard
Paperback
Programación profesional de Gleam: Cómo diseñar sistemas a gran escala con precisión de tipos.
Boozman, Richard
Paperback
Nim para principiantes: Cómo aprender un lenguaje flexible y moderno que combine conceptos de Python y Ada
Boozman, Richard
Paperback
Guía intermedia de Lua: Cómo crear scripts de juegos y extender aplicaciones con Lua
Boozman, Richard
Paperback
Lua para principiantes: Cómo aprender scripting ligero para juegos, aplicaciones y sistemas embebidos
Boozman, Richard
Paperback
Guía intermedia de OCaml: Cómo aplicar OCaml en finanzas, análisis y herramientas de software
Boozman, Richard
Paperback
Programación profesional en OCaml: Cómo diseñar sistemas de nivel industrial con OCaml
Boozman, Richard
Paperback
Guía intermedia de Nim: Cómo aplicar Nim en aplicaciones prácticas y de alto rendimiento
Boozman, Richard
Paperback
Programación profesional de Nim: Cómo diseñar sistemas complejos con flexibilidad y velocidad
Boozman, Richard
Paperback
OCaml para principiantes: Cómo aprender programación funcional e imperativa en la JVM
Boozman, Richard
Paperback
Professionelle NimProgrammierung: Wie man komplexe Systeme flexibel und schnell entwickelt
Boozman, Richard
Paperback
Nim für Anfänger: Wie man eine flexible, moderne Sprache erlernt, die Python- und Ada-Konzepte vereint
Boozman, Richard
Paperback
Nim-Leitfaden für Fortgeschrittene: Wie man Nim für praktische und leistungsstarke Anwendungen einsetzt
Boozman, Richard
Paperback
OCaml für Anfänger: Wie man funktionale und imperative Programmierung auf der JVM lernt
Boozman, Richard
Paperback
OCaml-Leitfaden für Fortgeschrittene: Anwendung von OCaml in den Bereichen Finanzen, Analyse und Softwareentwicklung
Boozman, Richard
Paperback
Professionelle OCamlProgrammierung: Wie man mit OCaml Systeme für den industriellen Einsatz entwickelt
Boozman, Richard
Paperback
Gleam für Anfänger: Wie man die BEAM VM mit einer typsicheren Sprache für nebenläufige Systeme nutzen kann
Boozman, Richard
Paperback
Gleam Leitfaden für Fortgeschrittene: Wie man mit Zuversicht parallele und fehlertolerante Systeme entwickelt
Boozman, Richard
Paperback
Professionelle Gleam-Programmierung: Wie man großtechnische Systeme mit typsicherer Präzision entwickelt
Boozman, Richard
Paperback
Lua-Leitfaden für Fortgeschrittene: Wie man Spielskripte erstellt und Anwendungen mit Lua erweitert
Boozman, Richard
Paperback
Der 30-Tage-Python-Sprint: Ein Leitfaden für Anfänger zur professionellen Kompetenz
Boozman, Richard
Paperback
El sprint de Python de 30 días: Guía para principiantes sobre competencia profesional
Boozman, Richard
Paperback
La lógica del código: Piensa como un programador antes de escribir una sola línea
Boozman, Richard
Paperback
LLM-Ingenieur: Erstellen, optimieren und bereitstellen Sie produktionsreife KI-Anwendungen mit Python und modernen LLMs.
Boozman, Richard
Paperback
Analytics Engineer: dbt, SQL Modeling, and Modern Data Transformation for Business Intelligence Teams
Boozman, Richard
Paperback
Belohnen Und Lernen: Praktisches Reinforcement Learning für autonome Agenten, Spiele und Robotersteuerung
Boozman, Richard
Paperback
Verantwortungsbewusst Es Design: KI-Sicherheit, Ausrichtung und Vertrauensentwicklung für produktive maschinelle Lernsysteme
Boozman, Richard
Paperback
Aus Daten Lernen: Statistisches maschinelles Lernen mit Python: Regression, Klassifizierung und neuronale Netze
Boozman, Richard
Paperback
Abschliessen: Mobile Sicherheitstechnik: Schutz von iOS- und Android-Anwendungen vor realen Bedrohungen
Boozman, Richard
Paperback
Flutter Everywhere: Cross-Platform Mobile Engineering for iOS, Android, and Web from a Single Codebase
Boozman, Richard
Paperback
Native by Nature: Building High-Performance, Native-Feel Mobile Apps with React Native and Expo
Boozman, Richard
Paperback
Compose Yourself: Modern Android UI Development with Jetpack Compose, State Management, and Animation
Boozman, Richard
Paperback
Lock It Down: Mobile Security Engineering: Protecting iOS and Android Applications from Real Threats
Boozman, Richard
Paperback
One Codebase, Every Platform: Kotlin Multiplatform in Production: Sharing Business Logic Across iOS, Android, and Web
Boozman, Richard
Paperback
Spark in the Wild: Big Data Processing, Streaming Analytics, and Machine Learning Pipelines at Scale
Boozman, Richard
Paperback
Kotlin Craft: Professionelle Android-Entwicklung: Architektur, Coroutinen und Produktionspraktiken
Boozman, Richard
Paperback
Also in
Programming
Interactive Theorem Proving and Program Development: Coq'art: The Calculus of Inductive Constructions
Castéran, Pierre
Bertot, Yves
Hardcover
The Legend of Zelda(tm) Tears of the Kingdom - The Complete Official Guide: Collector's Edition
Piggyback
Hardcover
Python Crash Course, 3rd Edition: A Hands-On, Project-Based Introduction to Programming
Matthes, Eric
Paperback
Accelerate: The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations
Humble, Jez
Kim, Gene
Forsgren Phd, Nicole
Paperback
The Legend of Zelda(tm) Tears of the Kingdom - The Complete Official Guide: Standard Edition
Piggyback
Paperback
Crisis Engineering: Time-Tested Tools for Turning Chaos Into Clarity
Nitze, Marina
Weaver, Matthew
Dickerson, Mikey
Paperback
Embedded Systems with ARM Cortex-M Microcontrollers in Assembly Language and C: Fourth Edition
Zhu, Yifeng
Paperback
Fundamentals of Software Architecture: A Modern Engineering Approach
Richards, Mark
Ford, Neal
Paperback
Vibe Coding: Building Production-Grade Software with Genai, Chat, Agents, and Beyond
Yegge, Steve
Kim, Gene
Paperback
Make: Electronics: Learning by Discovery: A Hands-On Primer for the New Electronics Enthusiast
Platt, Charles
Paperback
AI Projects with Raspberry Pi: High-Performance Artificial Intelligence for Robotics, Security, Home Automation, and Vision
Jepson, Brian
Hattersley, Lucy
Paperback
The Manager's Path: A Guide for Tech Leaders Navigating Growth and Change
Fournier, Camille
Paperback
I Have an App Idea: The Essential Guide to Building an App Without Tech Skills
Spann, Amanda
Paperback
The Return on AI: From Promise to Profit in the Age of Intelligent Business
Mittal, Ashwin
Sawhney, Mohanbir
Paperback
The Devops Handbook, 2nd Edition: How to Create World-Class Agility, Reliability, & Security in Technology Organizations
Humble, Jez
Debois, Patrick
Kim, Gene
Paperback
Building Applications with AI Agents: Designing and Implementing Multiagent Systems
Albada, Michael
Paperback
The Developer's Guide to AI: A Field Guide for the Working Developer
Thompson, Danny
Orshalick, Jacob
Reghunadh, Jerry M.
Paperback
The Staff Engineer's Path: A Guide for Individual Contributors Navigating Growth and Change
Reilly, Tanya
Paperback
Cracking the Coding Interview: 189 Programming Questions and Solutions
McDowell, Gayle Laakmann
Paperback
Software Architecture: The Hard Parts: Modern Trade-Off Analyses for Distributed Architectures
Sadalage, Pramod
Ford, Neal
Richards, Mark
Paperback
Learning Web Design: A Beginner's Guide to Html, Css, Javascript, and Web Images
Robbins, Jennifer
Paperback
The New AI Cold War: Liberty vs. Tyranny in the Age of Machine Empires
Maginnis, Ltc Robert L.
Paperback
Observability Engineering: Achieving Production Excellence
Fong-Jones, Liz
Miranda, George
Majors, Charity
Paperback
Forecasting: Principles and Practice, the Pythonic Way
Hyndman, Rob J.
Athanasopoulos, George
Paperback
Building Safer Technology: A Field Guide to Failing Well
Zunger, Yonatan
Coles, Neil
Kissner, Lea
Paperback
SQL QuickStart Guide: The Simplified Beginner's Guide to Managing, Analyzing, and Manipulating Data With SQL
Shields, Walter
Hardcover
The Pragmatic Programmer: Your Journey to Mastery, 20th Anniversary Edition
Thomas, David
Hunt, Andrew
Hardcover
Linux Basics for Hackers, 2nd Edition: Getting Started with Networking, Scripting, and Security in Kali
Occupytheweb
Paperback
Grokking Algorithms, Second Edition: An Illustrated Guide for Programmers and Other Curious People
Bhargava, Aditya Y.
Paperback
Generative AI Design Patterns: Solutions to Common Challenges When Building Genai Agents and Applications
Lakshmanan, Valliappa
Hapke, Hannes
Paperback
The Official Raspberry Pi Handbook 2026: Astounding Projects with Raspberry Pi Computers
Makers of Raspberry Pi Official Magazine, The
Paperback
Head First Design Patterns: Building Extensible and Maintainable Object-Oriented Software
Robson, Elisabeth
Freeman, Eric
Paperback
Concrete Mathematics: A Foundation for Computer Science
Graham, Ronald
Patashnik, Oren
Knuth, Donald
Hardcover
Learning Domain-Driven Design: Aligning Software Architecture and Business Strategy
Khononov, Vlad
Paperback
Arduino Programming for Beginners: A Comprehensive Beginner's Guide to Learn the Realms of Arduino Programming from A-Z
Protosmith, Ada
Paperback
The Software Architect Elevator: Redefining the Architect's Role in the Digital Enterprise
Hohpe, Gregor
Paperback
Frictionless: 7 Steps to Remove Barriers, Unlock Value, and Outpace Your Competition in the AI Era
Noda, Abi
Forsgren, Nicole
Paperback
Learning Generative AI Tools for Excel: Speed Up Your Everyday Tasks with Microsoft Excel, Copilot, Chatgpt, and Beyond
Duca, Angelica Lo
Paperback
Building AI-Powered Products: The Essential Guide to AI and Genai Product Management
Nika, Marily
Paperback
Debugging: The 9 Indispensable Rules for Finding Even the Most Elusive Software and Hardware Problems
Agans, David J.
Paperback
A Human's Guide to Machine Intelligence: How Algorithms Are Shaping Our Lives and How We Can Stay in Control
Hosanagar, Kartik
Paperback
Learning Php, MySQL & JavaScript: A Step-By-Step Guide to Creating Dynamic Websites
Nixon, Robin
Paperback
Build an AI Agent (from Scratch): Agents That Reason, Plan, and ACT Autonomously
Hur, Jungjun
Song, Younghee
Paperback
Software Engineering at Google: Lessons Learned from Programming Over Time
Manshreck, Tom
Wright, Hyrum
Winters, Titus
Paperback
Living a Jewish Life, Revised and Updated: Jewish Traditions, Customs, and Values for Today's Families
Diamant, Anita
Cooper, Howard
Paperback
Python Programming for Young Coders: A Hands-On, Project-Based Introduction to Coding for Beginners, Kids, and Teens
Pandey, Anand
Paperback
Computer Science from Scratch: Building Interpreters, Art, Emulators and ML in Python
Kopec, David
Paperback
Adobe Photoshop, 2nd Edition: A Complete Course and Compendium of Features
Laskevitch, Stephen
Paperback
Programming Massively Parallel Processors: A Hands-On Approach
El Hajj, Izzat
Hwu, Wen-Mei W.
Kirk, David B.
Paperback
Signals & Levers: Systems Thinking Tools to Unblock Software Delivery
Tosi, Joel
Hendrickson, Elisabeth
Paperback
Kubernetes in Action, Second Edition: Deploying and Managing Containers and Cloud-Native Applications
Luksa, Marko
Conner, Kevin
Paperback
Articulating Design Decisions: Communicate with Stakeholders, Keep Your Sanity, and Deliver the Best User Experience
Greever, Tom
Paperback
Embedded Systems with ARM Cortex-M Microcontrollers in Assembly Language and C: Third Edition
Zhu, Yifeng
Paperback
Prompt Engineering for Llms: The Art and Science of Building Large Language Model-Based Applications
Ziegler, Albert
Berryman, John
Paperback
Electronic Music and Sound Design - Theory and Practice with Max 9 - Volume 1 (Fifth Edition)
Cipriani, Alessandro
Maurizio, Giri
Paperback