Sign-In
Cart
Continuous-Time Markov Decision Processes: Borel Space Models and General Control Strategies
Hardcover
Series: Probability Theory and Stochastic Modelling, Book 97
ApplicationsGeneral ScienceProbability & Statistics
ISBN13: 9783030549862
Publisher: Springer Nature
Published: Nov 10 2020
Pages: 583
Weight: 2.25
Height: 1.31 Width: 6.14 Depth: 9.21
Language: English
This book offers a systematic and rigorous treatment of continuous-time Markov decision processes, covering both theory and possible applications to queueing systems, epidemiology, finance, and other fields. Unlike most books on the subject, much attention is paid to problems with functional constraints and the realizability of strategies.
Three major methods of investigations are presented, based on dynamic programming, linear programming, and reduction to discrete-time problems. Although the main focus is on models with total (discounted or undiscounted) cost criteria, models with average cost criteria and with impulsive controls are also discussed in depth.
The book is self-contained. A separate chapter is devoted to Markov pure jump processes and the appendices collect the requisite background on real analysis and applied probability. All the statements in the main text are proved in detail.
Researchers and graduate students in applied probability, operational research, statistics and engineering will find this monograph interesting, useful and valuable.
1 different editions
Also available
Continuous-Time Markov Decision Processes: Borel Space Models and General Control Strategies
Piunovskiy, Alexey
Zhang, Yi
Paperback
Also in
Probability & Statistics
The Art of Uncertainty: How to Navigate Chance, Ignorance, Risk and Luck
Spiegelhalter, David
Hardcover
Freakonomics Revised and Expanded Edition: A Rogue Economist Explores the Hidden Side of Everything
Dubner, Stephen J.
Levitt, Steven D.
Paperback
How the World Really Works: The Science Behind How We Got Here and Where We're Going
Smil, Vaclav
Paperback
AP Statistics Premium, 2026: Prep Book with 9 Practice Tests + Comprehensive Review + Online Practice
Sternstein, Martin
Paperback
R for Data Science: Import, Tidy, Transform, Visualize, and Model Data
Cetinkaya-Rundel, Mine
Wickham, Hadley
Grolemund, Garrett
Paperback
Stat 208 Statistical Thinking: A Book for Stat 208 at Virginia Commonwealth University
Street IV, W. Scott
Durfee, Becky
Paperback
Graph Paper Composition Notebook: Quad Ruled 5x5, Grid Paper for Students in Math and Science
Wizo, Math
Paperback
Introduction to Statistics: An Intuitive Guide for Analyzing Data and Unlocking Discoveries
Frost, Jim
Paperback
The Random Universe: How Models and Probability Help Us Make Sense of the Cosmos
Jaffe, Andrew H.
Hardcover
An Introduction to Statistical Learning: With Applications in R
Witten, Daniela
Hastie, Trevor
James, Gareth
Paperback
Mathematical Foundations of Quantum Computing: A Scaffolding Approach
Cheng, Ran
Lee, Peter
Yu, James
Hardcover
Scale: The Universal Laws of Life, Growth, and Death in Organisms, Cities, and Companies
West, Geoffrey
Paperback
How the World Really Works: The Science Behind How We Got Here and Where We're Going
Smil, Vaclav
Hardcover
Social Statistics for a Diverse Society
Leon-Guerrero, Anna
Frankfort-Nachmias, Chava
Davis, Georgiann
Paperback
Schaum's Outline of Probability, Random Variables, and Random Processes, Fourth Edition
Hsu, Hwei P.
Paperback
Building Generative AI Agents: Using Langgraph, Autogen, and Crewai
Taulli, Tom
Deshmukh, Gaurav
Paperback
AI for Robotics: Toward Embodied and General Intelligence in the Physical World
Gopalakrishnan, Keerthana
Imran, Alishba
Paperback
Probably Overthinking It: How to Use Data to Answer Questions, Avoid Statistical Traps, and Make Better Decisions
Downey, Allen B.
Hardcover
The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Second Edition
Tibshirani, Robert
Hastie, Trevor
Friedman, Jerome
Hardcover
The End of Average: Unlocking Our Potential by Embracing What Makes Us Different
Rose, Todd
Paperback
Mathletics: How Gamblers, Managers, and Fans Use Mathematics in Sports, Second Edition
Pelechrinis, Konstantinos
Winston, Wayne L.
Nestler, Scott
Paperback
The Master Algorithm: How the Quest for the Ultimate Learning Machine Will Remake Our World
Domingos, Pedro
Paperback
High School Probability: A New Perspective: For Any High School Curriculum, Including IB, A-Level, AP, and More
Seeds, Spring
Paperback
Advanced Statistics in Research: Reading, Understanding, and Writing Up Data Analysis Results
Hatcher, Larry
Paperback
The Only Math Book You'll Ever Need, Revised Edition: Hundreds of Easy Solutions and Shortcuts for Mastering Everyday Numbers
Kogelman, Stanley
Heller, Barbara R.
Paperback
Regression Analysis: An Intuitive Guide for Using and Interpreting Linear Models
Frost, Jim
Paperback
Probably the Best Book on Statistics Ever Written: How to Beat the Odds and Make Better Decisions
Shapira, Haim
Hardcover
Schaum's Outline of Probability and Statistics, 4th Edition: 897 Solved Problems + 20 Videos
Srinivasan, R. Alu
Spiegel, Murray R.
Schiller, John J.
Paperback
Machine Learning Q and AI: 30 Essential Questions and Answers on Machine Learning and AI
Raschka, Sebastian
Paperback
Mathematical Foundations of Quantum Computing: A Scaffolding Approach
Cheng, Ran
Lee, Peter
Yu, James
Paperback
Learning Statistics with Jamovi: A Tutorial for Beginners in Statistical Analysis
Foxcroft, David
Navarro, Danielle
Paperback
Making ChatGPT Work for You: Getting the Most Out of Generative AI as a Non-Techie
Evelyn, Lydia
Paperback
Practical Uncertainty: Useful Ideas in Decision-Making, Risk, Randomness, & AI
Pishro-Nik, Hossein
Paperback
Large Language Models: A Deep Dive: Bridging Theory and Practice
Keenan, Kevin
Somers, Garrett
Kamath, Uday
Hardcover
Probably Overthinking It: How to Use Data to Answer Questions, Avoid Statistical Traps, and Make Better Decisions
Downey, Allen B.
Paperback
Mathematical Modeling and Computation in Finance: With Exercises and Python and MATLAB Computer Codes
Oosterlee, Cornelis W.
Grzelak, Lech A.
Paperback
Student's Solutions Guide for Introduction to Probability, Statistics, and Random Processes
Pishro-Nik, Hossein
Paperback
The Theory That Would Not Die: How Bayes' Rule Cracked the Enigma Code, Hunted Down Russian Submarines, and Emerged Triumphant from Two Centuries of C
McGrayne, Sharon Bertsch
Paperback
Modern Time Series Forecasting with Python - Second Edition: Industry-ready machine learning and deep learning time series analysis with PyTorch and p
Tackes, Jeffrey
Joseph, Manu
Paperback
The Design Inference: Eliminating Chance through Small Probabilities
Dembski, William A.
Ewert, Winston
Hardcover
SQL Server 2025 Unveiled: The Ai-Ready Enterprise Database with Microsoft Fabric Integration
Ward, Bob
Paperback
Databricks Data Intelligence Platform: Unlocking the Genai Revolution
Yip, Jason
Gupta, Nikhil
Paperback
Designing Human-Centric AI Experiences: Applied UX Design for Artificial Intelligence
Kore, Akshay
Paperback
Design and Analysis of Experiments by Douglas Montgomery: A Supplement for Using JMP
Rushing, Heath
Karl, Andrew
Wisnowski, James
Paperback
The Unfinished Game: Pascal, Fermat, and the Seventeenth-Century Letter That Made the World Modern
Devlin, Keith
Paperback
The ESRI Guide to GIS Analysis, Volume 2: Spatial Measurements and Statistics
Mitchell, Andy
Griffin, Lauren Scott
Paperback
Challenging Mathematical Problems with Elementary Solutions, Vol. I: Volume 1
Yaglom, I. M.
Yaglom, A. M.
Paperback
Damned Lies and Statistics: Untangling Numbers from the Media, Politicians, and Activists
Best, Joel
Hardcover
Challenging Mathematical Problems with Elementary Solutions, Vol. II: Volume 2
Yaglom, A. M.
Yaglom, I. M.
Paperback
Telling Time Workbook: Practice Reading and Draw the Hand on the Clocks One Hour Half Hour 15 5 1 Minutes
Winny, Renzo
Paperback
Freakonomics REV Ed: A Rogue Economist Explores the Hidden Side of Everything (Large Print Edition)
Dubner, Stephen J.
Levitt, Steven D.
Paperback
Guesstimation: Solving the World's Problems on the Back of a Cocktail Napkin
Weinstein, Lawrence
Adam, John
Paperback
Enterprise Guide for Implementing Generative AI and Agentic AI: A Practical Guide to Developing, Deploying, and Operationalizing Ai-Driven Application
Bhattacharya, Rahul
Sinha, Vikas
Edward, Shakuntala Gupta
Paperback
An Introduction to Statistical Learning: With Applications in Python
Hastie, Trevor
James, Gareth
Witten, Daniela
Paperback
Lies, Damn Lies, and Statistics: The Manipulation of Public Opinion in America
Wheeler, Michael
Paperback
Introduction to Statistics: An Intuitive Guide for Analyzing Data and Unlocking Discoveries
Frost, Jim
Hardcover
Statistical Quantitative Methods in Finance: From Theory to Quantitative Portfolio Management
Ahlawat, Samit
Paperback
A Practical Guide for Mastering Generative AI Applications Using Amazon Bedrock: .
Bhattacharjee, Avik
Hardcover
An Introduction to Statistical Learning: With Applications in Python
Hastie, Trevor
James, Gareth
Witten, Daniela
Hardcover
Reliability: Probabilistic Models and Statistical Methods, Third Edition
Leemis, Lawrence M.
Paperback
Bayesian Analysis with Python - Third Edition: A practical guide to probabilistic modeling
Martin, Osvaldo
Hardcover
Practice Test for the COGAT Grade 5 Level 11: CogAT Test Prep Grade 5: Cognitive Abilities Test Form 7 and 8 for 5th Grade
Origins Publications
Paperback
Optimization and Learning Via Stochastic Gradient Search
Vázquez-Abad, Felisa
Heidergott, Bernd
Hardcover
Bayesian Analysis with Python - Third Edition: A practical guide to probabilistic modeling
Martin, Osvaldo
Paperback
Problems and Solutions in Stochastic Calculus with Applications
Klebaner, Fima C.
Albin, Patrik
Hamza, Kais
Paperback
Bayesian Models: A Statistical Primer for Ecologists, 2nd Edition
Hobbs, N. Thompson
Hooten, Mevin B.
Hardcover