Sign-In
Cart
Gradient Descent, Stochastic Optimization, and Other Tales
Paperback
Please select the version of book you would like to purchase.
ISBN10: 9994981552
ISBN13: 9789994981557
Publisher: Eliva Pr
Published: Jul 22 2022
Pages: 96
Weight: 0.31
Height: 0.20 Width: 6.00 Depth: 9.00
Language: English
ISBN13: 9789994981557
Publisher: Eliva Pr
Published: Jul 22 2022
Pages: 96
Weight: 0.31
Height: 0.20 Width: 6.00 Depth: 9.00
Language: English
The goal of this book is to debunk and dispel the magic behind the black-box optimizers and stochastic optimizers. It aims to build a solid foundation on how and why the techniques work. This manuscript crystallizes this knowledge by deriving from simple intuitions, the mathematics behind the strategies. This book doesn't shy away from addressing both the formal and informal aspects of gradient descent and stochastic optimization methods. By doing so, it hopes to provide readers with a deeper understanding of these techniques as well as the when, the how and the why of applying these algorithms.
Gradient descent is one of the most popular algorithms to perform optimization and by far the most common way to optimize machine learning tasks. Its stochastic version receives attention in recent years, and this is particularly true for optimizing deep neural networks. In deep neural networks, the gradient followed by a single sample or a batch of samples is employed to save computational resources and escape from saddle points. In 1951, Robbins and Monro published A stochastic approximation method, one of the first modern treatments on stochastic optimization that estimates local gradients with a new batch of samples. And now, stochastic optimization has become a core technology in machine learning, largely due to the development of the back propagation algorithm in fitting a neural network. The sole aim of this article is to give a self-contained introduction to concepts and mathematical tools in gradient descent and stochastic optimization.
Also from
Lu, Jun
Also in
General Mathematics
The Golden Number: Pythagorean Rites and Rhythms in the Development of Western Civilization
Ghyka, Matila C.
Hardcover
From $24.99
How It All Works: All Scientific Laws and Phenomena Illustrated & Demonstrated
Clegg, Brian
Dant, Adam
Hardcover
$14.99
Hidden Figures: The American Dream and the Untold Story of the Black Women Mathematicians Who Helped Win the Space Race
Shetterly, Margot Lee
Paperback
From $10.99
Six Easy Pieces: Essentials of Physics Explained by Its Most Brilliant Teacher
Feynman, Richard P.
Leighton, Robert B.
Sands, Matthew
Paperback
$15.29
Oliver Byrne's Elements of Euclid: The First Six Books with Coloured Diagrams and Symbols (Art Meets Science Edition)
Art Meets Science
Hardcover
$59.99
Math Practice Workbook Grades 6-8: 1000+ Questions You Need to Kill in Middle School by Brain Hunter Prep
Brain Hunter Prep
Paperback
$18.99
Who Gets What -- And Why: The New Economics of Matchmaking and Market Design
Roth, Alvin E.
Paperback
$17.10
The Art of Strategy: A Game Theorist's Guide to Success in Business and Life
Nalebuff, Barry J.
Dixit, Avinash K.
Paperback
$18.99
Tomoko Fuse's Amazing Kusudama Origami: Geometric Paper Art from Japan's Leading Expert!
Fuse, Tomoko
Paperback
$18.04
WorkKeys Study Guide and Practice Test Questions: ACT WorkKeys Exam Prep and Review Book with Applied Mathematics, Locating Information, and Reading f
Trivium Exam Prep Team
Paperback
$39.99
Mastering Essential Math Skills Book 2, Bilingual Edition - English/Spanish
Fisher, Richard W.
Paperback
$80.81
Hidden Figures: The American Dream and the Untold Story of the Black Women Mathematicians Who Helped Win the Space Race
Shetterly, Margot Lee
Hardcover
$28.49
The Great Math War: How Three Brilliant Minds Fought for the Foundations of Mathematics
Bardi, Jason Socrates
Hardcover
From $19.99
Counting with - Contando Con Frida (Bilingual: English/Spanish): A Bilingual Counting Book
Rodriguez, Patty
Stein, Ariana
Hardcover
$9.49
Math Games with Bad Drawings: 75 1/4 Simple, Challenging, Go-Anywhere Games--And Why They Matter
Orlin, Ben
Hardcover
$28.50
Quadrivium: The Four Classical Liberal Arts of Number, Geometry, Music, & Cosmology
Lundy, Miranda
Ashton, Anthony
Sutton, Daud
Hardcover
$24.70
Hidden Figures: The American Dream and the Untold Story of the Black Women Mathematicians Who Helped Win the Space Race
Shetterly, Margot Lee
Paperback
$18.99
Graph Paper Composition Notebook: Quad Ruled 5x5, Grid Paper for Students in Math and Science
Wizo, Math
Paperback
$8.74
50 Math Tricks That Will Change Your Life: Mentally Solve the Impossible in Seconds
Zakowich, Tanya
Paperback
$20.89
Vector: A Surprising Story of Space, Time, and Mathematical Transformation
Arianrhod, Robyn
Hardcover
$26.60
Quantum Mechanics Done Right: The Shortest Path from Novice to Researcher
Freericks, James K.
Hardcover
$83.99
Touchthinklearn: Numbers: (Board Books for Baby Learners, Touch Feel Books for Children)
Deneux, Xavier
Hardcover
$16.14
New York Post Fiendish Sudoku: The Official Utterly Addictive Number-Placing Puzzle
Gould, Wayne
Paperback
$10.99
A Mind for Numbers: How to Excel at Math and Science (Even If You Flunked Algebra)
Oakley, Barbara
Paperback
$19.00
ALEKS Math Placement Test Prep 2026: 1200 Verified Practice Questions with Accurate Answer Keys, Step-by-Step Worked Solutions, and Clear Concept Expl
Publishing, Kettford
Reid, Marcus L.
Paperback
$27.56
Partial Differential Equations: Theory and Completely Solved Problems
Leonard, I. E.
Van Roessel, H.
Hillen, T.
Paperback
$42.49
Secrets of Mental Math: The Mathemagician's Guide to Lightning Calculation and Amazing Math Tricks
Benjamin, Arthur
Shermer, Michael
Paperback
$20.90
Un Elefante: Numbers / Números (Bilingual: English/Spanish): A Bilingual Counting Book
Rodriguez, Patty
Stein, Ariana
Hardcover
$10.44
Sudoku for Brain Fitness: 90-Day Challenge to Sharpen the Mind and Strengthen Cognitive Skills
Clontz, Steven
Demyanovich, Julie
Paperback
$12.34
Building Thinking Classrooms in Mathematics, Grades K-12: 14 Teaching Practices for Enhancing Learning
Liljedahl, Peter
Paperback
$24.99
Concrete Mathematics: A Foundation for Computer Science
Knuth, Donald
Graham, Ronald
Patashnik, Oren
Hardcover
$107.99
Talentos Ocultos: La Genialidad No Tiene Color. La Fuerza No Tiene Género. El Valor No Tiene Límite.
Shetterly, Margot Lee
Paperback
$16.14
180 Days(r) Math for First Grade: Practice, Assess, Diagnose
Smith, Jodene
Smith, Jodene Lynn
Paperback
$22.79
Number Systems and the Foundations of Analysis
Mathematics
Mendelson, Elliott
Dudley, Underwood
Paperback
$18.95
The Beginner's Guide to Constructing the Universe: The Mathematical Archetypes of Nature, Art, and Science
Schneider, Michael S.
Paperback
$18.99
Medical Dosage Calculations for Dummies
Depuente, Vanessa
Schoenborn, Barry
Snyder, Richard W.
Paperback
$33.24
300 Math CrossWords: Addition, Subtraction, Multiplication, and Division Math CrossWords Grade 3 and Above, Age +8
Martin, Josh Ortiz
Paperback
$9.59
Math Fundamentals 3 - Geometry: For Middle School - A Quickstudy Laminated Reference Guide
Expolog
Warren, Peggy
Wright, Susan
Other
$6.60
Reaching for the Extreme: How the Quest for the Biggest, Fewest, and Weirdest Makes Math
Stewart, Ian
Hardcover
$28.45
Sciencia: Mathematics, Physics, Chemistry, Biology, and Astronomy for All
Watkins, Matthew
Betts, Moff
Tweed, Matt
Hardcover
From $12.99
30 More Math Mysteries Kids Can't Resist!: Quick & Clever Mysteries That Boost Problem-Solving Skills
Lee, Martin
Paperback
$12.34