Sign-In
Cart
Build a Large Language Model (from Scratch)
Paperback
Series: From Scratch
Please select the version of book you would like to purchase.
ISBN10: 1633437167
ISBN13: 9781633437166
Publisher: Manning Publications
Published: Oct 29 2024
Pages: 368
Weight: 1.35
Height: 0.90 Width: 7.40 Depth: 9.20
Language: English
ISBN13: 9781633437166
Publisher: Manning Publications
Published: Oct 29 2024
Pages: 368
Weight: 1.35
Height: 0.90 Width: 7.40 Depth: 9.20
Language: English
How to implement LLM attention mechanisms and GPT-style transformers. In Build a Large Language Model (from Scratch) bestselling author Sebastian Raschka guides you step by step through creating your own LLM. Each stage is explained with clear text, diagrams, and examples. You'll go from the initial design and creation, to pretraining on a general corpus, and on to fine-tuning for specific tasks. Build a Large Language Model (from Scratch) teaches you how to: - Plan and code all the parts of an LLM
- Prepare a dataset suitable for LLM training
- Fine-tune LLMs for text classification and with your own data
- Use human feedback to ensure your LLM follows instructions
- Load pretrained weights into an LLM Build a Large Language Model (from Scratch) takes you inside the AI black box to tinker with the internal systems that power generative AI. As you work through each key stage of LLM creation, you'll develop an in-depth understanding of how LLMs work, their limitations, and their customization methods. Your LLM can be developed on an ordinary laptop, and used as your own personal assistant. Purchase of the print book includes a free eBook in PDF and ePub formats from Manning Publications. About the technology Physicist Richard P. Feynman reportedly said, I don't understand anything I can't build. Based on this same powerful principle, bestselling author Sebastian Raschka guides you step by step as you build a GPT-style LLM that you can run on your laptop. This is an engaging book that covers each stage of the process, from planning and coding to training and fine-tuning. About the book Build a Large Language Model (From Scratch) is a practical and eminently-satisfying hands-on journey into the foundations of generative AI. Without relying on any existing LLM libraries, you'll code a base model, evolve it into a text classifier, and ultimately create a chatbot that can follow your conversational instructions. And you'll really understand it because you built it yourself! What's inside - Plan and code an LLM comparable to GPT-2
- Load pretrained weights
- Construct a complete training pipeline
- Fine-tune your LLM for text classification
- Develop LLMs that follow human instructions About the reader Readers need intermediate Python skills and some knowledge of machine learning. The LLM you create will run on any modern laptop and can optionally utilize GPUs. About the author Sebastian Raschka is a Staff Research Engineer at Lightning AI, where he works on LLM research and develops open-source software. The technical editor on this book was David Caswell. Table of Contents 1 Understanding large language models
2 Working with text data
3 Coding attention mechanisms
4 Implementing a GPT model from scratch to generate text
5 Pretraining on unlabeled data
6 Fine-tuning for classification
7 Fine-tuning to follow instructions
A Introduction to PyTorch
B References and further reading
C Exercise solutions
D Adding bells and whistles to the training loop
E Parameter-efficient fine-tuning with LoRA
- Fine-tune LLMs for text classification and with your own data
- Use human feedback to ensure your LLM follows instructions
- Load pretrained weights into an LLM Build a Large Language Model (from Scratch) takes you inside the AI black box to tinker with the internal systems that power generative AI. As you work through each key stage of LLM creation, you'll develop an in-depth understanding of how LLMs work, their limitations, and their customization methods. Your LLM can be developed on an ordinary laptop, and used as your own personal assistant. Purchase of the print book includes a free eBook in PDF and ePub formats from Manning Publications. About the technology Physicist Richard P. Feynman reportedly said, I don't understand anything I can't build. Based on this same powerful principle, bestselling author Sebastian Raschka guides you step by step as you build a GPT-style LLM that you can run on your laptop. This is an engaging book that covers each stage of the process, from planning and coding to training and fine-tuning. About the book Build a Large Language Model (From Scratch) is a practical and eminently-satisfying hands-on journey into the foundations of generative AI. Without relying on any existing LLM libraries, you'll code a base model, evolve it into a text classifier, and ultimately create a chatbot that can follow your conversational instructions. And you'll really understand it because you built it yourself! What's inside - Plan and code an LLM comparable to GPT-2
- Load pretrained weights
- Construct a complete training pipeline
- Fine-tune your LLM for text classification
- Develop LLMs that follow human instructions About the reader Readers need intermediate Python skills and some knowledge of machine learning. The LLM you create will run on any modern laptop and can optionally utilize GPUs. About the author Sebastian Raschka is a Staff Research Engineer at Lightning AI, where he works on LLM research and develops open-source software. The technical editor on this book was David Caswell. Table of Contents 1 Understanding large language models
2 Working with text data
3 Coding attention mechanisms
4 Implementing a GPT model from scratch to generate text
5 Pretraining on unlabeled data
6 Fine-tuning for classification
7 Fine-tuning to follow instructions
A Introduction to PyTorch
B References and further reading
C Exercise solutions
D Adding bells and whistles to the training loop
E Parameter-efficient fine-tuning with LoRA
Also from
Raschka, Sebastian
Machine Learning with PyTorch and Scikit-Learn: Develop machine learning and deep learning models with Python
Liu, Yuxi (Hayden)
Mirjalili, Vahid
Raschka, Sebastian
Paperback
$68.74
Machine Learning with PyTorch and Scikit-Learn: Develop machine learning and deep learning models with Python
Raschka, Sebastian
Liu, Yuxi (Hayden)
Hardcover
$99.99
Machine Learning Q and AI: 30 Essential Questions and Answers on Machine Learning and AI
Raschka, Sebastian
Paperback
$47.49
Python Machine Learning: Machine Learning and Deep Learning with Python, scikit-learn, and TensorFlow 2
Raschka, Sebastian
Mirjalili, Vahid
Paperback
$68.74
Python Machine Learning - Second Edition: Machine Learning and Deep Learning with Python, scikit-learn, and TensorFlow
Mirjalili, Vahid
Raschka, Sebastian
Paperback
$54.99
Python Machine Learning: Unlock deeper insights into Machine Leaning with this vital guide to cutting-edge predictive analytics
Raschka, Sebastian
Paperback
$48.99
Also in
General Computers
This Program Is Brought to You by . . .: Distributing Television News Online
Braun, Joshua A.
Paperback
$51.00
The Year in Tech, 2026: The Insights You Need from Harvard Business Review
Webb, Amy
Review, Harvard Business
Anthony, Scott D.
Paperback
$21.80
If Anyone Builds It, Everyone Dies: Why Superhuman AI Would Kill Us All
Yudkowsky, Eliezer
Soares, Nate
Hardcover
$27.00
The AI Con: How to Fight Big Tech's Hype and Create the Future We Want
Bender, Emily M.
Hanna, Alex
Hardcover
$30.40
Hbr's 10 Must Reads on AI (with Bonus Article How to Win with Machine Learning by Ajay Agrawal, Joshua Gans, and AVI Goldfarb)
Review, Harvard Business
Davenport, Thomas H.
Iansiti, Marco
Paperback
$23.70
Fans First: Change The Game, Break the Rules & Create an Unforgettable Experience
Cole, Jesse
Paperback
$18.99
Generative Ai: The Insights You Need from Harvard Business Review
Cremer, David De
Review, Harvard Business
Mollick, Ethan
Paperback
$21.80
The Technological Republic: Hard Power, Soft Belief, and the Future of the West
Zamiska, Nicholas W.
Karp, Alexander C.
Hardcover
From $17.99
Minecraft: Roll for Adventure: The Temple of the Charged Creeper
Forbeck, Marty
Forbeck, Matt
Hardcover
$19.00
The Experimentation Machine: Finding Product-Market Fit in the Age of AI
Bussgang, Jeffrey J.
Hardcover
$40.00
Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems
Kleppmann, Martin
Paperback
From $29.99
AI Snake Oil: What Artificial Intelligence Can Do, What It Can't, and How to Tell the Difference
Narayanan, Arvind
Kapoor, Sayash
Hardcover
$23.70
Minecraft: Guide Collection 4-Book Boxed Set (Updated): Survival (Updated), Creative (Updated), Redstone (Updated), Combat
Mojang Ab
The Official Minecraft Team
Hardcover
$50.31
The Black Swan: Second Edition: The Impact of the Highly Improbable: With a New Section: On Robustness and Fragility
Taleb, Nassim Nicholas
Paperback
$19.00
Atlas of AI: Power, Politics, and the Planetary Costs of Artificial Intelligence
Crawford, Kate
Paperback
$19.00
Python Crash Course, 3rd Edition: A Hands-On, Project-Based Introduction to Programming
Matthes, Eric
Paperback
$44.99
The Fourth Intelligence Revolution: The Future of Espionage and the Battle to Save America
Vinci, Anthony
Hardcover
$29.44
The Thinking Machine: Jensen Huang, Nvidia, and the World's Most Coveted Microchip
Witt, Stephen
Hardcover
$27.00
You to the Power of Two: Redefining Human Potential in the Age of Identic AI
Bradley, Joseph
Tapscott, Don
Hardcover
$33.20
Digital Ethics in the Age of AI: Navigating the ethical frontier today and beyond
Mehan, Julie
Paperback
$47.99
AI for Educators: Learning Strategies, Teacher Efficiencies, and a Vision for an Artificial Intelligence Future
Miller, Matt
Paperback
$23.70
Artificial Intelligence All-In-One for Dummies
Mueller, John Paul
Minnick, Chris
Massaron, Luca
Paperback
$47.49
Hands-On Large Language Models: Language Understanding and Generation
Alammar, Jay
Grootendorst, Maarten
Paperback
$75.99
More Human: How the Power of AI Can Transform the Way You Lead
Hougaard, Rasmus
Carter, Jacqueline
Hardcover
$30.40
The Coming Wave: Technology, Power, and the Twenty-First Century's Greatest Dilemma
Suleyman, Mustafa
Hardcover
From $12.99
The Podcast Pantheon: 101 Podcasts That Changed How We Listen--From Wtf to Serial
Malin, Sean
Hardcover
$23.70
AI Made Simple. Results Made Real.: An Executive's Guide to Partnering with the Future
Perley, Kathleen
Paperback
$19.00
AI Snake Oil: What Artificial Intelligence Can Do, What It Can't, and How to Tell the Difference
Narayanan, Arvind
Kapoor, Sayash
Paperback
$16.10
Tubes: A Journey to the Center of the Internet with a New Introduction by the Author
Blum, Andrew
Paperback
$19.99
Artificial Intelligence: A Guide for Thinking Humans (with a New Preface)
Mitchell, Melanie
Paperback
$19.00
The Cybernetic Society: How Humans and Machines Will Shape the Future Together
Husain, Amir
Hardcover
$28.50
Beast in the Machine: How Robotics and AI Will Transform Warfare and the Future of Human Conflict
Dougherty, George M.
Hardcover
$28.45
Microsoft 365 Excel All-In-One for Dummies
Ringstrom, David H.
Alexander, Michael
Kusleika, Dick
Paperback
$42.74
Laptops for Seniors in Easy Steps, 9th Edition: Covers All Laptops with the Windows 11 2024 Update
Vandome, Nick
Paperback
$23.74
AP Computer Science Principles Premium, 2026: Prep Book with 6 Practice Tests + Comprehensive Review + Online Practice
Reichelson, Seth
Paperback
$28.49
What Is Intelligence?: Lessons from AI about Evolution, Computing, and Minds
Aguera Y. Arcas, Blaise
Paperback
$35.10
The Death of Expertise: The Campaign Against Established Knowledge and Why It Matters
Nichols, Tom
Paperback
$17.98
Minecraft: Exploded Builds: Medieval Fortress: An Official Mojang Book
Mojang Ab
The Official Minecraft Team
Paperback
$11.39
RHCSA Red Hat Enterprise Linux 9: Training and Exam Preparation Guide (EX200), Third Edition
Ghori, Asghar
Paperback
$57.60
The Emergent Mind: How Intelligence Arises in People and Machines
Suri, Gaurav
McClelland, Jay
Hardcover
$30.40
Embedded Systems with ARM Cortex-M Microcontrollers in Assembly Language and C: Fourth Edition
Zhu, Yifeng
Paperback
$86.88
AI Valley: Microsoft, Google, and the Trillion-Dollar Race to Cash in on Artificial Intelligence
Rivlin, Gary
Hardcover
$30.40
The Year in Tech, 2025: The Insights You Need from Harvard Business Review
Webb, Amy
Farri, Elisa
Review, Harvard Business
Paperback
$21.80
Fundamentals of Data Engineering: Plan and Build Robust Data Systems
Reis, Joe
Housley, Matt
Paperback
$75.99
The Devops Handbook, 2nd Edition: How to Create World-Class Agility, Reliability, & Security in Technology Organizations
Humble, Jez
Debois, Patrick
Kim, Gene
Paperback
$36.10
The Magic of Code: How Digital Language Created and Connects Our World--And Shapes Our Future
Arbesman, Samuel
Hardcover
$27.00
Mindmasters: The Data-Driven Science of Predicting and Changing Human Behavior
Matz, Sandra
Hardcover
$28.50
Designing Machine Learning Systems: An Iterative Process for Production-Ready Applications
Huyen, Chip
Paperback
$62.69
Building Applications with AI Agents: Designing and Implementing Multiagent Systems
Albada, Michael
Paperback
$75.99
The AI Ultimatum: Preparing for a World of Intelligent Machines and Radical Transformation
Brown, Steve
Hill, Paul
Paperback
$25.00
Me, My Customer, and AI: The New Rules of Entrepreneurship
Werdelin, Henrik
Thorne, Nicholas
Hardcover
$24.70
R for Data Science: Import, Tidy, Transform, Visualize, and Model Data
Wickham, Hadley
Grolemund, Garrett
Cetinkaya-Rundel, Mine
Paperback
$75.99
Fundamentals of Software Architecture: A Modern Engineering Approach
Richards, Mark
Ford, Neal
Paperback
$75.99
Prompt Engineering for Generative AI: Future-Proof Inputs for Reliable AI Outputs
Taylor, Mike
Phoenix, James
Paperback
$75.99
Practical Charts: The Essential Guide to Creating Clear, Compelling Charts for Reports and Presentations
Desbarats, Nicholas P.
Paperback
$40.84
Graphic Artists Guild Handbook, 17th Edition: Pricing & Ethical Guidelines
The Graphic Artists Guild
Paperback
$49.46
Hands-On Machine Learning with Scikit-Learn and Pytorch: Concepts, Tools, and Techniques to Build Intelligent Systems
Géron, Aurélien
Paperback
$85.49
AI for the Authentic Leader: How to Communicate More Effectively Without Losing Your Humanity
Shapira, Allison
Hardcover
$23.74
Building AI-Powered Products: The Essential Guide to AI and Genai Product Management
Nika, Marily
Paperback
$42.74
AI for Life: 100+ Ways to Use Artificial Intelligence to Make Your Life Easier, More Productive...and More Fun!
Quillian, Celia
Paperback
$16.14
Verified: How to Think Straight, Get Duped Less, and Make Better Decisions about What to Believe Online
Caulfield, Mike
Wineburg, Sam
Paperback
$13.30
Mastering Video Content Creation: A Practical Guide to Social Media Growth with Expertly Shot and Edited Posts
Espejo, Justin
Paperback
$21.59
AI for Business: The Beginner's Fast Track to ChatGPT for Productivity, Profit, and Growth (2 books in 1)
Grant, Russel
Paperback
$25.64
The Chaos Machine: The Inside Story of How Social Media Rewired Our Minds and Our World
Fisher, Max
Paperback
$21.84
Isc2 Cissp Certified Information Systems Security Professional Official Study Guide & Practice Tests Bundle
Stewart, James Michael
Gibson, Darril
Chapple, Mike
Paperback
$114.00
The Mechanic and the Luddite: A Ruthless Criticism of Technology and Capitalism
Sadowski, Jathan
Paperback
$29.94
Software Architecture: The Hard Parts: Modern Trade-Off Analyses for Distributed Architectures
Sadalage, Pramod
Ford, Neal
Richards, Mark
Paperback
$75.99
Rewiring Your Mind for AI: How to Think, Work, and Thrive in the Age of Intelligence
Wood, David a.
Paperback
$28.45
AI with Intention: Principles and Action Steps for Teachers and School Leaders
Frontier, Tony
Paperback
$32.25
Rewiring Democracy: How AI Will Transform Our Politics, Government, and Citizenship
Sanders, Nathan E.
Schneier, Bruce
Hardcover
$26.96
Prompt Engineering for Llms: The Art and Science of Building Large Language Model-Based Applications
Berryman, John
Ziegler, Albert
Paperback
$75.99
Brave New Words: How AI Will Revolutionize Education (and Why That's a Good Thing)
Khan, Salman
Hardcover
$28.50
Prediction Machines, Updated and Expanded: The Simple Economics of Artificial Intelligence
Agrawal, Ajay
Gans, Joshua
Goldfarb, Avi
Hardcover
$30.40
Cloud Finops: Collaborative, Real-Time Cloud Value Decision Making
Storment, J. R.
Fuller, Mike
Paperback
$75.99
How To Think With AI: A Simple Guide to Boost Your Brain Power, Creativity, and Performance
McCauley, Alison
Hardcover
$23.74
Algorithms to Live by: The Computer Science of Human Decisions
Griffiths, Tom
Christian, Brian
Paperback
From $10.99