Sign-In
Cart
CUDA Programming in 21 Days: A Hands-On Course in C++ and Python
Paperback
Series: Learn Programming in 21 Days
Please select the version of book you would like to purchase.
ISBN13: 9798183777802
Publisher: Independently Published
Published: Jun 23 2026
Pages: 438
Weight: 1.65
Height: 0.89 Width: 7.50 Depth: 9.25
Language: English
Publisher: Independently Published
Published: Jun 23 2026
Pages: 438
Weight: 1.65
Height: 0.89 Width: 7.50 Depth: 9.25
Language: English
Twenty-one days. CUDA C++ is the spine; Python rides beside it the whole way. Every speed claim is measured, not just described.
A CUDA kernel that launches with no error is not one that ran correctly: CUDA errors are asynchronous and silent unless checked by hand. The bottleneck is rarely the arithmetic -- it is the PCIe link. Day 1 sets a NumPy loop beside a CuPy call, no kernel written yet, to show what a GPU buys and what the transfer costs. By Day 9 memory access is coalesced across a 32-thread warp instead of fought line by line. By Day 15 a naive reduction becomes a three-stage tiled one, benchmarked against numpy.sum. Day 21 closes with a performance checklist and a project brief of your own. 234 figures across 21 chapters, each a real measurement or hardware schematic. 187 listings in CUDA C++, CuPy or Numba, and every printed output captured from a real run on an RTX 4070. Every idea appears in more than one language, so a reader stuck in one can read the other. YOUR 21-DAY PATH Day 1 - Why a Different Kind of Chip? Parallel Thinking From the Ground Up: throughput versus latency, and a first measured speedup.Day 2 - Setting Up, Toolkit, Driver, and Your First Run: nvidia-smi, nvcc, and a device query.
Day 3 - Your First Kernel: __global__, the launch syntax, and SAXPY in CUDA C++, Numba and CuPy.
Day 4 - Threads, Blocks, and Grids: threadIdx, blockDim, the global-index formula, the grid-stride loop.
Day 5 - Vector Addition, End to End: the five steps of every CUDA program, cudaMalloc to a checked result.
Day 6 - Moving Data Between Host and Device: pinned memory, measured bandwidth, unified memory.
Day 7 - When Kernels Go Wrong, Error Checking and Debugging: cudaGetLastError, compute-sanitizer, a bug hunt.
Day 8 - The Memory Hierarchy: global, shared and constant memory, and a measured latency ladder.
Day 9 - Coalesced Memory Access: the 32-thread warp, strided versus contiguous access, array-of-structs.
Day 10 - Shared Memory and Tiling: shared memory as scratchpad, bank conflicts, a tiled matrix transpose.
Day 11 - Synchronization and Race Conditions: __syncthreads, a raced reduction, and why blocks cannot sync.
Day 12 - Warps and Divergence: lockstep execution, branch divergence, and a warp-shuffle reduction.
Day 13 - Occupancy and Launch Configuration: registers, shared memory, and the block-size sweet spot.
Day 14 - Measuring, Timing, Events, and a First Roofline: CUDA events, Nsight, the roofline model.
Day 15 - Reduction, the First Hard Kernel: tree reduction in shared memory, and a three-stage speedup ladder.
Day 16 - Atomics and a Histogram: atomicAdd, contention, and a privatized histogram.
Day 17 - Scan, Prefix Sum: inclusive and exclusive scan, and the Blelloch up-sweep, down-sweep.
Day 18 - Streams and Overlap: cudaMemcpyAsync, pinned memory, and a double-buffered pipeline.
Day 19 - Standing on Libraries, Thrust, cuBLAS, cuFFT, and CuPy: why a library beats a hand-rolled kernel.
Day 20 - A Real Application, Image Convolution on the GPU: a tiled kernel, verified against SciPy.
Day 21 - Wrapping Up and Where to Go Next: a performance checklist, the measure-first mindset, a project brief. WHO IT IS FOR. A programmer who knows a little C or C++ and a little Python, and wants to write, debug and profile a GPU kernel rather than just call one. You need a C/C++ compiler and Python with NumPy; a local GPU helps but is not required, since CuPy and Numba carry the ideas on a cloud notebook. No prior parallel programming, and no framework beyond the CUDA Toolkit. Download book source code from: {GitHub}/{mohammadnsaqib} 436 pages, black and white.
Day 3 - Your First Kernel: __global__, the launch syntax, and SAXPY in CUDA C++, Numba and CuPy.
Day 4 - Threads, Blocks, and Grids: threadIdx, blockDim, the global-index formula, the grid-stride loop.
Day 5 - Vector Addition, End to End: the five steps of every CUDA program, cudaMalloc to a checked result.
Day 6 - Moving Data Between Host and Device: pinned memory, measured bandwidth, unified memory.
Day 7 - When Kernels Go Wrong, Error Checking and Debugging: cudaGetLastError, compute-sanitizer, a bug hunt.
Day 8 - The Memory Hierarchy: global, shared and constant memory, and a measured latency ladder.
Day 9 - Coalesced Memory Access: the 32-thread warp, strided versus contiguous access, array-of-structs.
Day 10 - Shared Memory and Tiling: shared memory as scratchpad, bank conflicts, a tiled matrix transpose.
Day 11 - Synchronization and Race Conditions: __syncthreads, a raced reduction, and why blocks cannot sync.
Day 12 - Warps and Divergence: lockstep execution, branch divergence, and a warp-shuffle reduction.
Day 13 - Occupancy and Launch Configuration: registers, shared memory, and the block-size sweet spot.
Day 14 - Measuring, Timing, Events, and a First Roofline: CUDA events, Nsight, the roofline model.
Day 15 - Reduction, the First Hard Kernel: tree reduction in shared memory, and a three-stage speedup ladder.
Day 16 - Atomics and a Histogram: atomicAdd, contention, and a privatized histogram.
Day 17 - Scan, Prefix Sum: inclusive and exclusive scan, and the Blelloch up-sweep, down-sweep.
Day 18 - Streams and Overlap: cudaMemcpyAsync, pinned memory, and a double-buffered pipeline.
Day 19 - Standing on Libraries, Thrust, cuBLAS, cuFFT, and CuPy: why a library beats a hand-rolled kernel.
Day 20 - A Real Application, Image Convolution on the GPU: a tiled kernel, verified against SciPy.
Day 21 - Wrapping Up and Where to Go Next: a performance checklist, the measure-first mindset, a project brief. WHO IT IS FOR. A programmer who knows a little C or C++ and a little Python, and wants to write, debug and profile a GPU kernel rather than just call one. You need a C/C++ compiler and Python with NumPy; a local GPU helps but is not required, since CuPy and Numba carry the ideas on a cloud notebook. No prior parallel programming, and no framework beyond the CUDA Toolkit. Download book source code from: {GitHub}/{mohammadnsaqib} 436 pages, black and white.
Also from
Saqib, Mohammad
Building AI Agents in 21 Days: A Hands-On Course in Agentic Systems with Python and LLMs
Saqib, Mohammad
Paperback
$35.99
C23 Programming in 14 Days: Master Modern C and Systems Programming Through Daily Drills
Saqib, Mohammad
Paperback
$35.99
Bash 5.3 Programming in 14 Days: Master Defensive Shell Scripting for Sysadmin Automation through Daily Drills
Saqib, Mohammad
Paperback
$35.99
Fortran 2023 Programming in 14 Days: Master High-Performance Scientific Computing Through Daily Drills
Saqib, Mohammad
Paperback
$35.99
Also in
Programming
Interactive Theorem Proving and Program Development: Coq'art: The Calculus of Inductive Constructions
Castéran, Pierre
Bertot, Yves
Hardcover
$194.60
The Legend of Zelda(tm) Tears of the Kingdom - The Complete Official Guide: Collector's Edition
Piggyback
Hardcover
$42.74
Python Crash Course, 3rd Edition: A Hands-On, Project-Based Introduction to Programming
Matthes, Eric
Paperback
$44.99
The Legend of Zelda(tm) Tears of the Kingdom - The Complete Official Guide: Standard Edition
Piggyback
Paperback
$28.49
Crisis Engineering: Time-Tested Tools for Turning Chaos Into Clarity
Weaver, Matthew
Dickerson, Mikey
Nitze, Marina
Paperback
$18.99
Accelerate: The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations
Kim, Gene
Forsgren Phd, Nicole
Humble, Jez
Paperback
$20.89
Embedded Systems with ARM Cortex-M Microcontrollers in Assembly Language and C: Fourth Edition
Zhu, Yifeng
Paperback
$86.88
Fundamentals of Software Architecture: A Modern Engineering Approach
Ford, Neal
Richards, Mark
Paperback
$75.99
Make: Electronics: Learning by Discovery: A Hands-On Primer for the New Electronics Enthusiast
Platt, Charles
Paperback
$33.24
Vibe Coding: Building Production-Grade Software with Genai, Chat, Agents, and Beyond
Kim, Gene
Yegge, Steve
Paperback
$30.40
AI Projects with Raspberry Pi: High-Performance Artificial Intelligence for Robotics, Security, Home Automation, and Vision
Jepson, Brian
Hattersley, Lucy
Paperback
$21.84
The Manager's Path: A Guide for Tech Leaders Navigating Growth and Change
Fournier, Camille
Paperback
$37.99
I Have an App Idea: The Essential Guide to Building an App Without Tech Skills
Spann, Amanda
Paperback
$17.05
The Return on AI: From Promise to Profit in the Age of Intelligent Business
Sawhney, Mohanbir
Mittal, Ashwin
Paperback
$18.69
The Devops Handbook, 2nd Edition: How to Create World-Class Agility, Reliability, & Security in Technology Organizations
Debois, Patrick
Kim, Gene
Humble, Jez
Paperback
$36.10
The Staff Engineer's Path: A Guide for Individual Contributors Navigating Growth and Change
Reilly, Tanya
Paperback
$43.69
The Developer's Guide to AI: A Field Guide for the Working Developer
Reghunadh, Jerry M.
Thompson, Danny
Orshalick, Jacob
Paperback
$56.99
Building Applications with AI Agents: Designing and Implementing Multiagent Systems
Albada, Michael
Paperback
$75.99
Software Architecture: The Hard Parts: Modern Trade-Off Analyses for Distributed Architectures
Ford, Neal
Richards, Mark
Sadalage, Pramod
Paperback
$75.99
Cracking the Coding Interview: 189 Programming Questions and Solutions
McDowell, Gayle Laakmann
Paperback
From $24.99
Learning Web Design: A Beginner's Guide to Html, Css, Javascript, and Web Images
Robbins, Jennifer
Paperback
$75.99
Observability Engineering: Achieving Production Excellence
Majors, Charity
Fong-Jones, Liz
Miranda, George
Paperback
$66.49
Forecasting: Principles and Practice, the Pythonic Way
Athanasopoulos, George
Hyndman, Rob J.
Paperback
$76.00
The New AI Cold War: Liberty vs. Tyranny in the Age of Machine Empires
Maginnis, Ltc Robert L.
Paperback
$31.83
Linux Basics for Hackers, 2nd Edition: Getting Started with Networking, Scripting, and Security in Kali
Occupytheweb
Paperback
$37.99
Building Safer Technology: A Field Guide to Failing Well
Coles, Neil
Kissner, Lea
Zunger, Yonatan
Paperback
$65.99
The Pragmatic Programmer: Your Journey to Mastery, 20th Anniversary Edition
Thomas, David
Hunt, Andrew
Hardcover
$54.99
A Human's Guide to Machine Intelligence: How Algorithms Are Shaping Our Lives and How We Can Stay in Control
Hosanagar, Kartik
Paperback
$27.00
SQL QuickStart Guide: The Simplified Beginner's Guide to Managing, Analyzing, and Manipulating Data With SQL
Shields, Walter
Hardcover
$33.24
Grokking Algorithms, Second Edition: An Illustrated Guide for Programmers and Other Curious People
Bhargava, Aditya Y.
Paperback
$62.49
The Official Raspberry Pi Handbook 2026: Astounding Projects with Raspberry Pi Computers
Makers of Raspberry Pi Official Magazine, The
Paperback
$23.74
Generative AI Design Patterns: Solutions to Common Challenges When Building Genai Agents and Applications
Lakshmanan, Valliappa
Hapke, Hannes
Paperback
$75.99
Concrete Mathematics: A Foundation for Computer Science
Graham, Ronald
Patashnik, Oren
Knuth, Donald
Hardcover
$107.99
Learning Domain-Driven Design: Aligning Software Architecture and Business Strategy
Khononov, Vlad
Paperback
$62.69
Site Reliability Engineering: How Google Runs Production Systems
Leng, Christof
Beyer, Betsy
Jones, Chris
Paperback
$75.99
Head First Design Patterns: Building Extensible and Maintainable Object-Oriented Software
Robson, Elisabeth
Freeman, Eric
Paperback
$75.99
The Software Architect Elevator: Redefining the Architect's Role in the Digital Enterprise
Hohpe, Gregor
Paperback
$62.69
Learning Generative AI Tools for Excel: Speed Up Your Everyday Tasks with Microsoft Excel, Copilot, Chatgpt, and Beyond
Duca, Angelica Lo
Paperback
$56.99
Arduino Programming for Beginners: A Comprehensive Beginner's Guide to Learn the Realms of Arduino Programming from A-Z
Protosmith, Ada
Paperback
$18.99
Building AI-Powered Products: The Essential Guide to AI and Genai Product Management
Nika, Marily
Paperback
$42.74
Building Resilient Distributed Systems: Patterns and Practices for Stable Software
Newman, Sam
Paperback
$66.49
Frictionless: 7 Steps to Remove Barriers, Unlock Value, and Outpace Your Competition in the AI Era
Noda, Abi
Forsgren, Nicole
Paperback
$33.74
Build an AI Agent (from Scratch): Agents That Reason, Plan, and ACT Autonomously
Hur, Jungjun
Song, Younghee
Paperback
$74.99
Debugging: The 9 Indispensable Rules for Finding Even the Most Elusive Software and Hardware Problems
Agans, David J.
Paperback
$18.99
AI Agents with MCP: Model Context Protocol for Building Clients, Servers, and End-To-End Agents
Stratis, Kyle
Paperback
$75.99
Signals & Levers: Systems Thinking Tools to Unblock Software Delivery
Hendrickson, Elisabeth
Tosi, Joel
Paperback
$30.40
Software Engineering at Google: Lessons Learned from Programming Over Time
Wright, Hyrum
Winters, Titus
Manshreck, Tom
Paperback
$62.69
Learning Php, MySQL & JavaScript: A Step-By-Step Guide to Creating Dynamic Websites
Nixon, Robin
Paperback
$56.99
Living a Jewish Life, Revised and Updated: Jewish Traditions, Customs, and Values for Today's Families
Cooper, Howard
Diamant, Anita
Paperback
$22.79
Python Programming for Young Coders: A Hands-On, Project-Based Introduction to Coding for Beginners, Kids, and Teens
Pandey, Anand
Paperback
$20.57
Java in a Nutshell: A Desktop Quick Reference
Clark, Jason R.
Flanagan, David
Evans, Benjamin J.
Paperback
$66.49
Programming Massively Parallel Processors: A Hands-On Approach
El Hajj, Izzat
Hwu, Wen-Mei W.
Kirk, David B.
Paperback
$130.43
Computer Science from Scratch: Building Interpreters, Art, Emulators and ML in Python
Kopec, David
Paperback
$47.49
Adobe Photoshop, 2nd Edition: A Complete Course and Compendium of Features
Laskevitch, Stephen
Paperback
$52.20
Kubernetes in Action, Second Edition: Deploying and Managing Containers and Cloud-Native Applications
Conner, Kevin
Luksa, Marko
Paperback
$87.49
Electronic Music and Sound Design - Theory and Practice with Max 9 - Volume 1 (Fifth Edition)
Cipriani, Alessandro
Maurizio, Giri
Paperback
$76.85
Prompt Engineering for Llms: The Art and Science of Building Large Language Model-Based Applications
Ziegler, Albert
Berryman, John
Paperback
$75.99
Practical AI Security: A Hands-On Guide to Attacking, Defending, and Securing Modern AI Systems
Farlow, Harriet
Paperback
$56.99