Sign-In
Cart
Data Engineering Best Practices: Architect robust and cost-effective data solutions in the cloud era
Paperback
ISBN13: 9781803244983
Publisher: Packt Publishing
Published: Oct 11 2024
Pages: 550
Weight: 2.06
Height: 1.11 Width: 7.50 Depth: 9.25
Language: English
Explore modern data engineering techniques and best practices to build scalable, efficient, and future-proof data processing systems across cloud platforms
Key Features:
- Architect and engineer optimized data solutions in the cloud with best practices for performance and cost-effectiveness
- Explore design patterns and use cases to balance roles, technology choices, and processes for a future-proof design
- Learn from experts to avoid common pitfalls in data engineering projects
- Purchase of the print or Kindle book includes a free PDF eBook
Book Description:
Revolutionize your approach to data processing in the fast-paced business landscape with this essential guide to data engineering. Discover the power of scalable, efficient, and secure data solutions through expert guidance on data engineering principles and techniques. Written by two industry experts with over 60 years of combined experience, it offers deep insights into best practices, architecture, agile processes, and cloud-based pipelines.
You'll start by defining the challenges data engineers face and understand how this agile and future-proof comprehensive data solution architecture addresses them. As you explore the extensive toolkit, mastering the capabilities of various instruments, you'll gain the knowledge needed for independent research. Covering everything you need, right from data engineering fundamentals, the guide uses real-world examples to illustrate potential solutions. It elevates your skills to architect scalable data systems, implement agile development processes, and design cloud-based data pipelines. The book further equips you with the knowledge to harness serverless computing and microservices to build resilient data applications.
By the end, you'll be armed with the expertise to design and deliver high-performance data engineering solutions that are not only robust, efficient, and secure but also future-ready.
What You Will Learn:
- Architect scalable data solutions within a well-architected framework
- Implement agile software development processes tailored to your organization's needs
- Design cloud-based data pipelines for analytics, machine learning, and AI-ready data products
- Optimize data engineering capabilities to ensure performance and long-term business value
- Apply best practices for data security, privacy, and compliance
- Harness serverless computing and microservices to build resilient, scalable, and trustworthy data pipelines
Who this book is for:
If you are a data engineer, ETL developer, or big data engineer who wants to master the principles and techniques of data engineering, this book is for you. A basic understanding of data engineering concepts, ETL processes, and big data technologies is expected. This book is also for professionals who want to explore advanced data engineering practices, including scalable data solutions, agile software development, and cloud-based data processing pipelines.
Table of Contents
- Overview of the Business Problem Statement
- A Data Engineer's Journey - Background Challenges
- A Data Engineer's Journey - IT's Vision and Mission
- Architecture Principles
- Architecture Framework - Conceptual Architecture Best Practices
- Architecture Framework - Logical Architecture Best Practices
- Architecture Framework - Physical Architecture Best Practices
- Software Engineering Best Practice Considerations
- Key Considerations for Agile SDLC Best Practices
- Key Considerations for Quality Testing Best Practices
- Key Considerations for IT Operational Service Best Practices
(N.B. Please use the Read Sample option to see further chapters)
Also in
Databases
Seven Databases in Seven Weeks: A Guide to Modern Databases and the Nosql Movement
Perkins, Luc
Redmond, Eric
Wilson, Jim
Paperback
Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems
Riccomini, Chris
Kleppmann, Martin
Paperback
Fundamentals of Data Engineering: Plan and Build Robust Data Systems
Reis, Joe
Housley, Matt
Paperback
SQL QuickStart Guide: The Simplified Beginner's Guide to Managing, Analyzing, and Manipulating Data With SQL
Shields, Walter
Hardcover
The Data Warehouse Toolkit: The Definitive Guide to Dimensional Modeling
Kimball, Ralph
Ross, Margy
Paperback
Observability Engineering: Achieving Production Excellence
Majors, Charity
Fong-Jones, Liz
Miranda, George
Paperback
Introduction to Statistics: An Intuitive Guide for Analyzing Data and Unlocking Discoveries
Frost, Jim
Paperback
Learning Tableau 2025 - Sixth Edition: Leverage Tableau's newest features to revolutionize your data storytelling with AI-enhanced insights
Milligan, Joshua N.
Paperback
The Data Storyteller's Handbook: How to create business impact using data storytelling
Greenbrook, Kat
Paperback
Ai-Assisted Statistics for Data Scientists: 50+ Essential Concepts Using R and Python
Gedeck, Peter
Bruce, Peter
Bruce, Andrew
Paperback
Microsoft Power Bi Data Analyst Associate Study Guide: Prepare for the Pl-300 Exam and Use Power Bi Successfully
Turley, Paul
Paperback
Database Systems: Introduction to Databases and Data Warehouses, Edition 2.0
Jukic, Nenad
Vrbsky, Susan
Sharma, Abhishek
Paperback
SQL for Data Analysis: Advanced Techniques for Transforming Data Into Insights
Tanimura, Cathy
Paperback
Data Engineering Design Patterns: Recipes for Solving the Most Common Data Engineering Problems
Konieczny, Bartosz
Paperback
Data Engineering for Cybersecurity: Build Secure Data Pipelines with Free and Open-Source Tools
Bonifield, James
Paperback
High Performance Spark: Best Practices for Scaling and Optimizing Apache Spark
Warren, Rachel
Karau, Holden
Polak, Adi
Paperback
SQL Server 2025 Unveiled: The Ai-Ready Enterprise Database with Microsoft Fabric Integration
Ward, Bob
Paperback
SQL QuickStart Guide: The Simplified Beginner's Guide to Managing, Analyzing, and Manipulating Data With SQL
Shields, Walter
Paperback
Becoming a Data Head: How to Think, Speak, and Understand Data Science, Statistics, and Machine Learning
Goldmeier, Jordan
Gutman, Alex J.
Paperback
Databricks Certified Data Engineer Associate Study Guide: In-Depth Guidance and Practice
Alhussein, Derar
Paperback
Mathletics: How Gamblers, Managers, and Fans Use Mathematics in Sports, Second Edition
Pelechrinis, Konstantinos
Winston, Wayne L.
Nestler, Scott
Paperback
Kafka: The Definitive Guide: Real-Time Data and Stream Processing at Scale
Palino, Todd
Sivaram, Rajini
Shapira, Gwen
Paperback
Observability Engineering: Achieving Production Excellence
Miranda, George
Majors, Charity
Fong-Jones, Liz
Paperback
Product Operations: How successful companies build better products at scale
Perri, Melissa
Tilles, Denise
Paperback
Turning Data into Wisdom: How We Can Collaborate with Data to Change Ourselves, Our Organizations, and Even the World
Hanegan, Kevin
Paperback
Foundations of Deep Reinforcement Learning: Theory and Practice in Python
Keng, Wah Loon
Graesser, Laura
Paperback
Data Contracts: Developing Production-Grade Pipelines at Scale
Sanderson, Chad
Schmidt, B. E.
Freeman, Mark
Paperback
High Performance Python: Practical Performant Programming for Humans
Gorelick, Micha
Ozsvald, Ian
Paperback
Data Analytics Made Easy: Analyze and present data to make informed decisions without writing any code
Mauro, Andrea de
Paperback
Data Governance with Unity Catalog on Databricks: Implement Data and AI Governance with Databricks Data Intelligence Platform
Sreekumar, Kiran
Subbarao, Karthik
Paperback
SQL Cookbook: Query Solutions and Techniques for All SQL Users
Graaf, Robert de
Molinaro, Anthony
Paperback
Head First Algorithms and Data Structures: A Learner's Guide to the Foundations of Computing
Freeman, Eric
Robson, Elisabeth
Paperback
ML and Generative AI in the Data Lakehouse: Building and Deploying AI Applications at Scale
Haelen, Bennie
Paperback
Numerical Python: Scientific Computing and Data Science Applications with Numpy, Scipy and Matplotlib
Johansson, Robert
Paperback
Data Analytics & Visualization All-In-One for Dummies
McFedries, Paul
Massaron, Luca
Hyman, Jack A.
Paperback
Hands-On Data Analysis with Pandas: Efficiently perform data collection, wrangling, analysis, and visualization using Python
Molin, Stefanie
Paperback
Fusion Strategy: How Real-Time Data and AI Will Power the Industrial Future
Govindarajan, Vijay
Venkatraman, Venkat
Hardcover
Kafka for Architects: Event-Driven Architecture, Logs, Microservices, Real-Time Event Processing
Gorshkova, Katya
Paperback
The Data Center as a Computer: Designing Warehouse-Scale Machines
Hölzle, Urs
Ranganathan, Parthasarathy
Barroso, Luiz André
Hardcover
Databricks Data Intelligence Platform: Unlocking the Genai Revolution
Yip, Jason
Gupta, Nikhil
Paperback
SQL Server 2025 Query Performance Tuning: Troubleshoot and Optimize Query Performance
Fritchey, Grant
Paperback
Advanced Snowflake: Processing Data, Developing Applications, and Deploying ML Models at Scale
Ullah, Muhammad Fasih
Paperback
Learning Ransomware Response & Recovery: Stopping Ransomware One Restore at a Time
Saylor, Michael
Preston, W. Curtis
Paperback
Non-Invasive Data Governance: The Path of Least Resistance and Greatest Success
Seiner, Robert
Paperback
Ai-Ready Data Blueprints: From Raw Data to Ai-Driven Innovation
Sopirala, Srikanth
Shukla, Navnit
Pham, Kien
Paperback
Apache Iceberg: The Definitive Guide: Data Lakehouse Functionality, Performance, and Scalability on the Data Lake
Hughes, Jason
Merced, Alex
Shiran, Tomer
Paperback
Data and Reality: A Timeless Perspective on Perceiving and Managing Information in Our Imprecise World, 3rd Edition
Kent, William
Paperback
Football Analytics with Python & R: Learning Data Science Through the Lens of Sports
Erickson, Richard a.
Eager, Eric A.
Paperback
Mastering PostgreSQL 17 - Sixth Edition: Elevate your database skills with advanced deployment, optimization, and security strategies
Schönig, Hans-Jürgen
Paperback
Data Pipelines with Apache Airflow, Second Edition: Orchestration for Data and AI
Geusebroek, Kris
Ruiter, Julian de
Cabral, Ismael
Paperback
Apache Polaris: The Definitive Guide: Enriching Apache Iceberg Data Lakehouses with an Open Source Catalog
Madson, Andrew
Merced, Alex
Shiran, Tomer
Paperback
PostgreSQL 16 Administration Cookbook: Solve real-world Database Administration challenges with 180+ practical recipes and best practices
Angelakos, Jimmy
Ciolli, Gianni
Mejías, Boriss
Paperback
Delta Lake: The Definitive Guide: Modern Data Lakehouse Architectures with Data Lakes
Wentling, Tristen
Haines, Scott
Lee, Denny
Paperback
How to Interpret Data: Using Data to Improve Your Influence and Decision-Making
Kelly, Nicholas
Paperback
Practical Natural Language Processing: A Comprehensive Guide to Building Real-World Nlp Systems
Gupta, Anuj
Vajjala, Sowmya
Majumder, Bodhisattwa
Paperback
Apache Hudi: The Definitive Guide: Building Robust, Open, and High-Performing Data Lakehouses
Xu, Shiyan
Wason, Prashant
Saktheeswaran, Bhavani Sudha
Paperback
Learning Statistics with Jamovi: A Tutorial for Beginners in Statistical Analysis
Navarro, Danielle
Foxcroft, David
Paperback
Agile Data Warehouse Design: Collaborative Dimensional Modeling, from Whiteboard to Star Schema
Stagnitto, Jim
Corr, Lawrence
Paperback
Real-Time Intelligence with Microsoft Fabric: Empowering Data-Driven Decisions in the Era of AI
Geisler, Frank
Brattã S. Johan Ludvig
Paperback
Everybody Lies: Big Data, New Data, and What the Internet Can Tell Us about Who We Really Are
Stephens-Davidowitz, Seth
Paperback
Data Modeling with Microsoft Power BI: Self-Service and Enterprise Data Warehouse with Power BI
Ehrenmueller-Jensen, Markus
Paperback
Building Data Integration Solutions: Unifying Data for Enhanced Decision Making
Borthen, Jay
Paperback
The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Second Edition
Friedman, Jerome
Hastie, Trevor
Tibshirani, Robert
Hardcover
Learn PostgreSQL - Second Edition: Use, manage and build secure and scalable databases with PostgreSQL 16
Ferrari, Luca
Pirozzi, Enrico
Paperback
Unlocking Dbt: Design and Deploy Transformations in Your Cloud Data Warehouse
Dorsey, Dustin
Cyr, Cameron
Paperback
Analytics Engineering with SQL and Dbt: Building Meaningful Data Models at Scale
Machado, Rui
Russa, Hélder
Paperback
SQL for Data Analytics - Fourth Edition: Analyze data effectively, uncover insights and master advanced SQL for real-world applications
Goldwasser, Matt
Shan, Jun
Li, Haibin
Paperback
Big Data in Der Mobilität: Akteure, Geschäftsmodelle Und Nutzenpotenziale Für Die Welt Von Morgen
Knorre, Susanne
Müller-Peters, Horst
Gatzert, Nadine
Paperback
Data Science at the Command Line: Obtain, Scrub, Explore, and Model Data with UNIX Power Tools
Janssens, Jeroen
Paperback
Statistics Every Programmer Needs: Practical Python Implementations and Quantitative Methods
Sutton, Gary
Paperback
The Data Product Playbook: Designing and Delivering Data Products that Power Decisions, Analytics, and AI
Koenders, Willem
Paperback
Data Engineering with Python: Work with massive datasets to design data models and automate data pipelines using Python
Crickard, Paul
Paperback
Introduction to Statistics: An Intuitive Guide for Analyzing Data and Unlocking Discoveries
Frost, Jim
Hardcover
Data Governance Change Management: From Drift to Direction-Why Every Data Leader Needs an ANCHOR
Agrawal, Aakriti
Agrawal, Arvind
Paperback
Stream Processing with Apache Flink: Fundamentals, Implementation, and Operation of Streaming Applications
Kalavri, Vasiliki
Hueske, Fabian
Paperback
Natural Language Processing with Pytorch: Build Intelligent Language Applications Using Deep Learning
McMahan, Brian
Rao, Delip
Paperback
The Expert's Guide to SQL Server: Tuning, Security, and Cloud Migration
Andiappan, Petchikumar
Pasam, Venkata Reddy
Paperback
Microsoft Power Bi Visual Calculations: Simplifying Dax
Lelijveld, Marc
Ter Heerdt, Jeroen
Stikkelorum, Madzy
Paperback
SQL in a Week: A Fast-Track Guide to Writing and Optimizing Queries Like a Pro
Shah, Bhumika
Dubey, Devtosh
Paperback
R in Action, Third Edition: Data Analysis and Graphics with R and Tidyverse
Kabacoff, Robert I.
Paperback
SAP S/4hana Financial Accounting Certification Guide: Application Associate Exam
Pougkas, Stefanos
Paperback
A Friendly Guide to Data Science: Everything You Should Know about the Hottest Field in Tech
Vincent, Kelly P.
Paperback
iPhone 11 Pro Max Guide: Learn Step-By-Step How To Use Your iPhone Pro Max Step-By-Step
Meyer, Albrecht
Paperback
Blockchain Basics: A Non-Technical Introduction in 25 Steps + 5 Steps Beyond the Basics
Drescher, Daniel
Paperback
PostgreSQL Query Optimization: The Ultimate Guide to Building Efficient Queries
Database Expert
Dombrovskaya, Henrietta
Bailliekova, Anna
Paperback
Data Governance: The Definitive Guide: People, Processes, and Tools to Operationalize Data Trustworthiness
Lakshmanan, Valliappa
Eryurek, Evren
Gilad, Uri
Paperback
SQL for Data Scientists: A Beginner's Guide for Building Datasets for Analysis
Teate, Renee M. P.
Paperback
Data Science on the Google Cloud Platform: Implementing End-To-End Real-Time Data Pipelines: From Ingest to Machine Learning
Lakshmanan, Valliappa
Paperback
The Elephant in the Fridge: Guided Steps to Data Vault Success through Building Business-Centered Models
Giles, John
Paperback
Mastering PostgreSQL Administration: Internals, Operations, Monitoring, and Oracle Migration Strategies
Kadambari, Phani
Ravi Kumar, Y. V.
Samayam, Arun Kumar
Paperback
The Data Management Workbook: Practical Exercises for Better Organization, Storage and Use of Your Research Data
Briney, Kristin
Paperback
The Definitive Guide to Microsoft Fabric: From discovery to building a unified, secure, and scalable data platform
Beau, Emilie
Riehl, Jean-Pierre
Maneu, Christopher
Paperback