%{search_type} search results

• Preface.- List of symbols.- Introduction.- Part I: First-Rare-Event Times for Regularly Perturbed Semi-Markov Processes.- Flows of Rare Events for Regularly Perturbed Semi-Markov Processes.- Generalizations of Limit Theorems for First-Rare-Event Times.- First-Rare-Event Times for Perturbed Risk Processes.- First-Rare-Event Times for Perturbed Closed Queuing Systems.- First-Rare-Event Times for Perturbed M/M-Type Queuing Systems.- Part II: Hitting Times and Phase Space Reduction for Perturbed Semi-Markov Processes.- Asymptotically Comparable Functions.- Perturbed Semi-Markov Processes and Reduction of Phase Space.- Asymptotics of Hitting Times for Perturbed Semi-Markov Processes.- Asymptotics for Expectations of Hitting Times for Perturbed Semi-Markov Processes.- Generalizations and Examples of Limit Theorems for Hitting Times.- Limit Theorems for Randomly Stopped Stochastic Processes.- Methodological and Bibliographical Notes.- References.- Index.
• (source: Nielsen Book Data)

This book is the first volume of a two-volume monograph devoted to the study of limit and ergodic theorems for regularly and singularly perturbed Markov chains, semi-Markov processes, and multi-alternating regenerative processes with semi-Markov modulation. The first volume presents necessary and sufficient conditions for weak convergence for first-rare-event times and convergence in the topology J for first-rare-event processes defined on regularly perturbed finite Markov chains and semi-Markov processes. The text introduces new asymptotic recurrent algorithms of phase space reduction. It also addresses both effective conditions of weak convergence for distributions of hitting times as well as convergence of expectations of hitting times for regularly and singularly perturbed finite Markov chains and semi-Markov processes. The book also contains a comprehensive bibliography of major works in the field. It provides an effective reference for both graduate students as well as theoretical and applied researchers studying stochastic processes and their applications. .
(source: Nielsen Book Data)

• Tools.- Markov renewal processes and related processes.- First steps with PDMP.- Hitting time distribution.- Intensity of some marked point pocesses.- Generalized Kolmogorov equations.- A martingale approach.- Stability.- Numerical methods.- Switching Processes.- Tools.- Interarrival distribution with several Dirac measures.- Algorithm convergence's proof.
• (source: Nielsen Book Data)

This book is aimed at researchers, graduate students and engineers who would like to be initiated to Piecewise Deterministic Markov Processes (PDMPs). A PDMP models a deterministic mechanism modified by jumps that occur at random times. The fields of applications are numerous : insurance and risk, biology, communication networks, dependability, supply management, etc. Indeed, the PDMPs studied so far are in fact deterministic functions of CSMPs (Completed Semi-Markov Processes), i.e. semi-Markov processes completed to become Markov processes. This remark leads to considerably broaden the definition of PDMPs and allows their properties to be deduced from those of CSMPs, which are easier to grasp. Stability is studied within a very general framework. In the other chapters, the results become more accurate as the assumptions become more precise. Generalized Chapman-Kolmogorov equations lead to numerical schemes. The last chapter is an opening on processes for which the deterministic flow of the PDMP is replaced with a Markov process. Marked point processes play a key role throughout this book.
(source: Nielsen Book Data)

• Preface ix
• Acknowledgments xiii
• Introduction xv
• Part 1. Basic Methods 1
• Chapter 1. Preliminary Concepts 3
• 1.1. Introduction to random evolutions 3
• 1.2. Abstract potential operators 7
• 1.3. Markov processes: operator semigroups 11
• 1.4. Semi-Markov processes 14
• 1.5. Lumped Markov chains 17
• 1.6. Switched processes in Markov and semi-Markov media 19
• Chapter 2. Homogeneous Random Evolutions (HRE) and their Applications 23
• 2.1. Homogeneous random evolutions (HRE) 24
• 2.1.1. Definition and classification of HRE 24
• 2.1.2. Some examples of HRE 25
• 2.1.3. Martingale characterization of HRE 28
• 2.1.4. Analogue of Dynkin's formula for HRE 34
• 2.1.5. Boundary value problems for HRE 36
• 2.2. Limit theorems for HRE 37
• 2.2.1. Weak convergence of HRE 37
• 2.2.2. Averaging of HRE 39
• 2.2.3. Diffusion approximation of HRE 42
• 2.2.4. Averaging of REs in reducible phase space: merged HRE 45
• 2.2.5. Diffusion approximation of HRE in reducible phase space 48
• 2.2.6. Normal deviations of HRE 51
• 2.2.7. Rates of convergence in the limit theorems for HRE 53
• Part 2. Applications to Reliability, Random Motions, and Telegraph Processes 57
• Chapter 3. Asymptotic Analysis for Distributions of Markov, Semi-Markov and Random Evolutions 59
• 3.1. Asymptotic distribution of time to reach a level that is infinitely increasing by a family of semi-Markov processes on the set ? 61
• 3.2. Asymptotic inequalities for the distribution of the occupation time of a semi-Markov process in an increasing set of states 74
• 3.3. Asymptotic analysis of the occupation time distribution of an embedded semi-Markov process (with increasing states) in a diffusion process 77
• 3.4. Asymptotic analysis of a semigroup of operators of the singularly perturbed random evolution in semi-Markov media 82
• 3.5. Asymptotic expansion for distribution of random motion in Markov media under the Kac condition 90
• 3.5.1. The equation for the probability density of the particle position performing a random walk in ?n 90
• 3.5.2. Equation for the probability density of the particle position 91
• 3.5.3. Reduction of a singularly perturbed evolution equation to a regularly perturbed equation 93
• 3.6. Asymptotic estimation for application of the telegraph process as an alternative to the diffusion process in the Black-Scholes formula 96
• 3.6.1. Asymptotic expansion for the singularly perturbed random evolution in Markov media in case of disbalance 96
• 3.6.2. Application to an economic model of stock market 100
• Chapter 4. Random Switched Processes with Delay in Reflecting Boundaries 103
• 4.1. Stationary distribution of evolutionary switched processes in a Markov environment with delay in reflecting boundaries 104
• 4.2. Stationary distribution of switched process in semi-Markov media with delay in reflecting barriers 109
• 4.2.1. Infinitesimal operator of random evolution with semi-Markov switching 110
• 4.2.2. Stationary distribution of random evolution in semi-Markov media with delaying boundaries in balance case 113
• 4.2.3. Stationary distribution of random evolution in semi-Markov media with delaying boundaries 121
• 4.3. Stationary efficiency of a system with two unreliable subsystems in cascade and one buffer: the Markov case 124
• 4.3.1. Introduction 124
• 4.3.2. Stationary distribution of Markov stochastic evolutions 125
• 4.3.3. Stationary efficiency of a system with two unreliable subsystems in cascade and one buffer 129
• 4.3.4. Mathematical model 131
• 4.3.5. Main mathematical results 133
• 4.3.6. Numerical results for the symmetric case 138
• 4.4. Application of random evolutions with delaying barriers to modelling control of supply systems with feedback: the semi-Markov switching process 141
• 4.4.1. Estimation of stationary efficiency of one-phase system with a reservoir 141
• 4.4.2. Estimation of stationary efficiency of a production system with two unreliable supply lines 149
• Chapter 5. One-dimensional Random Motions in Markov and Semi-Markov Media 159
• 5.1. One-dimensional semi-Markov evolutions with general Erlang sojourn times 160
• 5.1.1. Mathematical model 160
• 5.1.2. Solution of PDEs with constant coefficients and derivability of functions ranged in commutative algebras 168
• 5.1.3. Infinite-dimensional case 171
• 5.1.4. The distribution of one-dimensional random evolutions in Erlang media 172
• 5.2. Distribution of limiting position of fading evolution 181
• 5.2.1. Distribution of random power series in cases of uniform and Erlang distributions 182
• 5.2.2. The distribution of the limiting position 190
• 5.3. Differential and integral equations for jump random motions 191
• 5.3.1. The Erlang jump telegraph process on a line 192
• 5.3.2. Examples 198
• 5.4. Estimation of the number of level crossings by the telegraph process 199
• 5.4.1. Estimation of the number of level crossings for the telegraph process in Kac's condition 202
• References 205
• Index 219
• Summary of Volume 2 221.
• (source: Nielsen Book Data)

This book is the first of two volumes on random motions in Markov and semi-Markov random environments. This first volume focuses on homogenous random motions. This volume consists of two parts, the first describing the basic concepts and methods that have been developed for random evolutions. These methods are the foundational tools used in both volumes, and this description includes many results in potential operators. Some techniques to find closed-form expressions in relevant applications are also presented. The second part deals with asymptotic results and presents a variety of applications, including random motion with different types of boundaries, the reliability of storage systems and solutions of partial differential equations with constant coefficients, using commutative algebra techniques. It also presents an alternative formulation to the Black-Scholes formula in finance, fading evolutions and telegraph processes, including jump telegraph processes and the estimation of the number of level crossings for telegraph processes.
(source: Nielsen Book Data)

• Preface ix
• Acknowledgments xiii
• Introduction xv
• Part 1. Higher-dimensional Random Motions and Interactive Particles 1
• Chapter 1. Random Motions in Higher Dimensions 3
• 1.1. Random motion at finite speed with semi-Markov switching directions process 5
• 1.1.1. Erlang-K-distributed direction alternations 7
• 1.1.2. Some properties of the random walk in a semi-Markov environment and its characteristic function 15
• 1.2. Random motion with uniformly distributed directions and random velocity 17
• 1.2.1. Renewal equation for the characteristic function of isotropic motion with random velocity in a semi-Markov media 17
• 1.2.2. One-dimensional case 20
• 1.2.3. Two-dimensional case 23
• 1.2.4. Three-dimensional case 23
• 1.2.5. Four-dimensional case 31
• 1.3. The distribution of random motion at non-constant velocity in semi-Markov media 32
• 1.3.1. Renewal equation for the characteristic function 34
• 1.3.2. Two-dimensional case 35
• 1.3.3. Three-dimensional case 37
• 1.3.4. Four-dimensional case 40
• 1.4. Goldstein-Kac telegraph equations and random flights in higher dimensions 43
• 1.4.1. Preliminaries about our modeling approach 45
• 1.4.2. Two-dimensional case 48
• 1.4.3. Three-dimensional case 51
• 1.4.4. Five-dimensional case 59
• 1.5. The jump telegraph process in Rn 62
• 1.5.1. The jump telegraph process in R3 63
• 1.5.2. Conclusions and final remarks 64
• Chapter 2. System of Interactive Particles with Markov and Semi-Markov Switching 67
• 2.1. Description of the Markov model 68
• 2.1.1. Distribution of the first meeting time of two telegraph processes 69
• 2.1.2. Estimate of the number of particle collisions 74
• 2.1.3. Free path times of a family of particles 76
• 2.1.4. Estimation of the number of particle collisions for systems with boundaries 78
• 2.1.5. Estimation of the number of particle collisions for systems without boundaries 83
• 2.2. Interaction of particles governed by generalized integrated telegraph processes: a semi-Markov case 87
• 2.2.1. Laplace transform of the distribution of the first collision of two particles 88
• 2.2.2. Semi-Markov case 91
• 2.2.3. Distribution of the first collision of two particles with finite expectation 95
• Part 2. Financial Applications 99
• Chapter 3. Asymptotic Estimation for Application of the Telegraph Process as an Alternative to the Diffusion Process in the Black-Scholes Formula 101
• 3.1. Asymptotic expansion for the singularly perturbed random evolution in Markov media in the case of disbalance 101
• 3.2. Application: Black-Scholes formula 106
• Chapter 4. Variance, Volatility, Covariance and Correlation Swaps for Financial Markets with Markov-modulated Volatilities 111
• 4.1. Volatility derivatives 111
• 4.1.1. Types of volatilities 111
• 4.1.2. Models for volatilities 113
• 4.1.3. Variance and volatility swaps 115
• 4.1.4. Covariance and correlation swaps 116
• 4.1.5. A brief literature review 117
• 4.2. Martingale representation of a Markov process 118
• 4.3. Variance and volatility swaps for financial markets with Markov-modulated stochastic volatilities 122
• 4.3.1. Pricing variance swaps 124
• 4.3.2. Pricing volatility swaps 124
• 4.4. Covariance and correlation swaps for two risky assets for financial markets with Markov-modulated stochastic volatilities 128
• 4.4.1. Pricing covariance swaps 128
• 4.4.2. Pricing correlation swaps 130
• 4.4.3. Correlation swap made simple 130
• 4.5. Example: variance, volatility, covariance and correlation swaps for stochastic volatility driven by two state continuous Markov chain 132
• 4.6. Numerical example 134
• 4.6.1. S&P 500: variance and volatility swaps 134
• 4.6.2. S&P 500 and NASDAQ-100: covariance and correlation swaps 135
• 4.7. Appendix 1 138
• 4.7.1. Correlation swaps: first-order correction 138
• Chapter 5. Modeling and Pricing of Variance, Volatility, Covariance and Correlation Swaps for Financial Markets with Semi-Markov Volatilities 143
• 5.1. Introduction 143
• 5.2. Martingale representation of semi-Markov processes 148
• 5.3. Variance and volatility swaps for financial markets with semi-Markov stochastic volatilities 151
• 5.3.1. Pricing of variance swaps 153
• 5.3.2. Pricing of volatility swaps 155
• 5.3.3. Numerical evaluation of variance and volatility swaps with semi-Markov volatility 158
• 5.4. Covariance and correlation swaps for two risky assets in financial markets with semi-Markov stochastic volatilities 159
• 5.4.1. Pricing of covariance swaps 160
• 5.4.2. Pricing of correlation swaps 162
• 5.5. Numerical evaluation of covariance and correlation swaps with semi-Markov stochastic volatility 164
• 5.6. Appendices 165
• 5.6.1. Appendix 1. Realized correlation: first-order correction 165
• 5.6.2. Appendix 2. Discussions of some extensions 169
• References 177
• Index 191
• Summary of Volume 1 193.
• (source: Nielsen Book Data)

This book is the second of two volumes on random motions in Markov and semi-Markov random environments. This second volume focuses on high-dimensional random motions. This volume consists of two parts. The first expands many of the results found in Volume 1 to higher dimensions. It presents new results on the random motion of the realistic three-dimensional case, which has so far been barely mentioned in the literature, and deals with the interaction of particles in Markov and semi-Markov media, which has, in contrast, been a topic of intense study. The second part contains applications of Markov and semi-Markov motions in mathematical finance. It includes applications of telegraph processes in modeling stock price dynamics and investigates the pricing of variance, volatility, covariance and correlation swaps with Markov volatility and the same pricing swaps with semi-Markov volatilities.
(source: Nielsen Book Data)

• Preface.- 1 Probability Review.- 2 Discrete-Time Markov Chains.- 3 Recurrence and Ergodicity.- 4 Long-Run Behavior.- 5 Discrete-Time Renewal Theory.- 6 Absorption and Passage Times.- 7 Lyapunov Functions and Martingales.- 8 Random Walks on Graphs.- 9 Convergence Rates.- 10 Markov Fields on Graphs.- 11 Monte Carlo Markov Chains.- 12 Non-homogeneous Markov Chains.- 13 Continuous-Time Markov Chains.- 14 Markovian Queueing Theory.- Appendices.- Bibliography.- Index.
• (source: Nielsen Book Data)

Primarily an introduction to the theory of stochastic processes at the undergraduate or beginning graduate level, the primary objective of this book is to initiate students in the art of stochastic modelling. However it is motivated by significant applications and progressively brings the student to the borders of contemporary research. Examples are from a wide range of domains, including operations research and electrical engineering. Researchers and students in these areas as well as in physics, biology and the social sciences will find this book of interest.
(source: Nielsen Book Data)

• PREFACE NOTATIONS LIST OF FIGURES LIST OF TABLES 1.RANDOM PROCESS 2.DISCRETE STATE SPACE MARKOV PROCESSES 3.SEMI-MARKOV PROCESS 4.CHARACTERISTICS OF SEMI-MARKOV PROCESS 5.PERTURBED SEMI-MARKOV PROCESSES 6.STOCHASTIC PROCESSES ASSOCIATED WITH SEMI-MARKOV PROCESS 7.MODELS OF RENEWABLE COLD STANDBY SYSTEM WITH REPAIR 8.SEMI-MARKOV MODELS OF MULTI-STAGE OPERATION 9.SEMI-MARKOV MODEL OF WORKING RATE PROCESS 10.MULTI-TASK OPERATION PROCESS 11.SEMI-MARKOV FAILURE RATE PROCESS 12.MODEL OF RENEWABLE SERIES SYSTEM 13.SIMPLE MODELS OF MAINTENANCE 14.MULTI-STATE PROCESS OF SYSTEM DAMAGE 15.MULTI-STATE SYSTEM WITH SEMI-MARKOV COMPONENTS 16.SEMI-MARKOV MAINTENANCE NET 17.SEMI-MARKOV DECISION PROCESSES 18.APPENDIX SUMMARY BIBLIOGRAPHY.
• (source: Nielsen Book Data)

Semi-Markov Processes: Applications in System Reliability and Maintenance is a modern view of discrete state space and continuous time semi-Markov processes and their applications in reliability and maintenance. The book explains how to construct semi-Markov models and discusses the different reliability parameters and characteristics that can be obtained from those models. The book is a useful resource for mathematicians, engineering practitioners, and PhD and MSc students who want to understand the basic concepts and results of semi-Markov process theory.
(source: Nielsen Book Data)

• Introduction
• 1 La notion de probabilité.-2 Variables aléatoires discrètes
• 2 Vecteurs aléatoires
• 4 Espérance conditionnelle
• 5 Information et entropie.
• 6 L'espérance comme intégrale
• 7 Suites de variablesaléatoires
• 8 Chaînes de Markov
• Solutions des exercices.-Bibliographie
• Index.

Ce cours, qui s'adresse aux étudiants des universités et des grandes écoles, donne les éléments de la théorie des probabilités utiles à la compréhension des modèles probabilistes de leurs spécialités respectives, ainsi que la pratique du calcul des probabilités nécessaire à l'exploitation de ces modèles. Cette initiation aux probabilités comporte trois degrés: le calcul des probabilités, la théorie des probabilités, les chaînes de Markov. La première partie du cours introduit les notions essentielles: événements, probabilité, variable aléatoire, probabilité conditionnelle, indépendance. L'accent est mis sur les outils de base (fonction génératrice, fonction caractéristique) et le calcul des probabilités (règles de Bayes, changement de variable, calcul sur les matrices de covariance et les vecteurs gaussiens). Un court chapitre est consacré à la notion d'entropie et à sa signification en théorie des communications et en physique statistique. Le seul prérequis pour cette première étape est une connaissance pratique des séries, de l'intégrale de Riemann et de l'algèbre matricielle. La deuxième partie concerne la théorie des probabilités proprement dite. Elle débute par un résumé motivé des résultats de la théorie de l'intégration de Lebesgue, qui fournit le cadre mathématique de la théorie axiomatique des probabilités et précise les points techniques laissés provisoirement dans l'ombre dans la première partie. Puis vient un chapitre où sont étudiées les différentes notions de convergence, et dans lequel sont présentés les deux sommets de la théorie, la loi forte des grands nombres et le théorème de la limite gaussienne. Le chapitre final, qui constitue à lui seul la troisième étape de l'initiation, traite des chaînes de Markov, la plus importante classe de processus stochastiques pour les applications. En fin de chaque chapitre se trouve une section d'exercices, la plupart corrigés, sauf ceux marqués d'un astérisque.

• 1 Introduction 2 Basic Bounds on Mixing Times 3 Advanced Functional Techniques 4 Evolving Set Methods 5 Lower Bounds on Mixing Times and their Consequences 6 Examples 7 Miscellaneous 8 Open Problems Acknowledgements References Appendix.
• (source: Nielsen Book Data)

This book begins with a gentle introduction to the analytical aspects of the theory of finite Markov chain mixing times and quickly ramps up to explain the latest developments in the topic. Several theorems are revisited and often derived in simpler, transparent ways, and illustrated with examples. The highlights include spectral, logarithmic Sobolev techniques, the evolving set methodology, and issues of nonreversibility. This is a comprehensive, well-written review of the subject that will be of interest to researchers and students in computer and mathematical sciences.
(source: Nielsen Book Data)

• Preface xi
• 1 Combinatorics 1 1.1 Binomial coefficients 1 1.1.1 Pascal triangle 1 1.1.2 Some properties of binomial coefficients 2 1.1.3 Generalized binomial coefficients and binomial series 3 1.1.4 Inversion formulas 4 1.1.5 Exercises 6 1.2 Sets, permutations and functions 8 1.2.1 Sets 8 1.2.2 Permutations 8 1.2.3 Multisets 10 1.2.4 Lists and functions 11 1.2.5 Injective functions 12 1.2.6 Monotone increasing functions 12 1.2.7 Monotone nondecreasing functions 13 1.2.8 Surjective functions 14 1.2.9 Exercises 16 1.3 Drawings 16 1.3.1 Ordered drawings 16 1.3.2 Simple drawings 17 1.3.3 Multiplicative property of drawings 17 1.3.4 Exercises 18 1.4 Grouping 19 1.4.1 Collocations of pairwise different objects 19 1.4.2 Collocations of identical objects 22 1.4.3 Multiplicative property 23 1.4.4 Collocations in statistical physics 24 1.4.5 Exercises 24
• 2 Probability measures 27 2.1 Elementary probability 28 2.1.1 Exercises 29 2.2 Basic facts 33 2.2.1 Events 34 2.2.2 Probability measures 36 2.2.3 Continuity of measures 37 2.2.4 Integral with respect to a measure 39 2.2.5 Probabilities on finite and denumerable sets 40 2.2.6 Probabilities on denumerable sets 42 2.2.7 Probabilities on uncountable sets 44 2.2.8 Exercises 46 2.3 Conditional probability 51 2.3.1 Definition 51 2.3.2 Bayes formula 52 2.3.3 Exercises 54 2.4 Inclusion exclusion principle 60 2.4.1 Exercises 63
• 3 Random variables 68 3.1 Random variables 68 3.1.1 Definitions 69 3.1.2 Expected value 75 3.1.3 Functions of random variables 77 3.1.4 Cavalieri formula 80 3.1.5 Variance 82 3.1.6 Markov and Chebyshev inequalities 82 3.1.7 Variational characterization of the median and of the expected value 83 3.1.8 Exercises 84 3.2 A few discrete distributions 91 3.2.1 Bernoulli distribution 91 3.2.2 Binomial distribution 91 3.2.3 Hypergeometric distribution 93 3.2.4 Negative binomial distribution 94 3.2.5 Poisson distribution 95 3.2.6 Geometric distribution 98 3.2.7 Exercises 101 3.3 Some absolutely continuous distributions 102 3.3.1 Uniform distribution 102 3.3.2 Normal distribution 104 3.3.3 Exponential distribution 106 3.3.4 Gamma distributions 108 3.3.5 Failure rate 110 3.3.6 Exercises 111
• 4 Vector valued random variables 113 4.1 Joint distribution 113 4.1.1 Joint and marginal distributions 114 4.1.2 Exercises 117 4.2 Covariance 120 4.2.1 Random variables with finite expected value and variance 120 4.2.2 Correlation coefficient 123 4.2.3 Exercises 123 4.3 Independent random variables 124 4.3.1 Independent events 124 4.3.2 Independent random variables 127 4.3.3 Independence of many random variables 128 4.3.4 Sum of independent random variables 130 4.3.5 Exercises 131 4.4 Sequences of independent random variables 140 4.4.1 Weak law of large numbers 140 4.4.2 Borel Cantelli lemma 142 4.4.3 Convergences of random variables 143 4.4.4 Strong law of large numbers 146 4.4.5 A few applications of the law of large numbers 152 4.4.6 Central limit theorem 159 4.4.7 Exercises 163
• 5 Discrete time Markov chains 168 5.1 Stochastic matrices 168 5.1.1 Definitions 169 5.1.2 Oriented graphs 170 5.1.3 Exercises 172 5.2 Markov chains 173 5.2.1 Stochastic processes 173 5.2.2 Transition matrices 174 5.2.3 Homogeneous processes 174 5.2.4 Markov chains 174 5.2.5 Canonical Markov chains 178 5.2.6 Exercises 181 5.3 Some characteristic parameters 187 5.3.1 Steps for a first visit 187 5.3.2 Probability of (at least) r visits 189 5.3.3 Recurrent and transient states 191 5.3.4 Mean first passage time 193 5.3.5 Hitting time and hitting probabilities 195 5.3.6 Exercises 198 5.4 Finite stochastic matrices 201 5.4.1 Canonical representation 201 5.4.2 States classification 203 5.4.3 Exercises 205 5.5 Regular stochastic matrices 206 5.5.1 Iterated maps 206 5.5.2 Existence of fixed points 209 5.5.3 Regular stochastic matrices 210 5.5.4 Characteristic parameters 218 5.5.5 Exercises 220 5.6 Ergodic property 222 5.6.1 Number of steps between consecutive visits 222 5.6.2 Ergodic theorem 224 5.6.3 Powers of irreducible stochastic matrices 226 5.6.4 Markov chain Monte Carlo 228 5.7 Renewal theorem 233 5.7.1 Periodicity 233 5.7.2 Renewal theorem 234 5.7.3 Exercises 239
• 6 An introduction to continuous time Markov chains 241 6.1 Poisson process 241 6.2 Continuous time Markov chains 246 6.2.1 Definitions 246 6.2.2 Continuous semigroups of stochastic matrices 248 6.2.3 Examples of right-continuous Markov chains 256 6.2.4 Holding times 259 Appendix A Power series 261 A.1 Basic properties 261 A.2 Product of series 263 A.3 Banach space valued power series 264 A.3.2 Exercises 267 Appendix B Measure and integration 270 B.1 Measures 270 B.1.1 Basic properties 270 B.1.2 Construction of measures 272 B.1.3 Exercises 279 B.2 Measurable functions and integration 279 B.2.1 Measurable functions 280 B.2.2 The integral 283 B.2.3 Properties of the integral 284 B.2.4 Cavalieri formula 286 B.2.5 Markov inequality 287 B.2.6 Null sets and the integral 287 B.2.7 Push forward of a measure 289 B.2.8 Exercises 290 B.3 Product measures and iterated integrals 294 B.3.1 Product measures 294 B.3.2 Reduction formulas 296 B.3.3 Exercises 297 B.4 Convergence theorems 298 B.4.1 Almost everywhere convergence 298 B.4.2 Strong convergence 300 B.4.3 Fatou lemma 301 B.4.4 Dominated convergence theorem 302 B.4.5 Absolute continuity of integrals 305 B.4.6 Differentiation of the integral 305 B.4.7 Weak convergence of measures 308 B.4.8 Exercises 312 Appendix C Systems of linear ordinary differential equations 313 C.1 Cauchy problem 313 C.1.1 Uniqueness 313 C.1.2 Existence 315 C.2 Efficient computation of eQt 317 C.2.1 Similarity methods 317 C.2.2 Putzer method 319 C.3 Continuous semigroups 321 References 324 Index 327.
• (source: Nielsen Book Data)

Provides an introduction to basic structures of probability with a view towards applications in information technology A First Course in Probability and Markov Chains presents an introduction to the basic elements in probability and focuses on two main areas. The first part explores notions and structures in probability, including combinatorics, probability measures, probability distributions, conditional probability, inclusion-exclusion formulas, random variables, dispersion indexes, independent random variables as well as weak and strong laws of large numbers and central limit theorem. In the second part of the book, focus is given to Discrete Time Discrete Markov Chains which is addressed together with an introduction to Poisson processes and Continuous Time Discrete Markov Chains. This book also looks at making use of measure theory notations that unify all the presentation, in particular avoiding the separate treatment of continuous and discrete distributions. A First Course in Probability and Markov Chains: * Presents the basic elements of probability. * Explores elementary probability with combinatorics, uniform probability, the inclusion-exclusion principle, independence and convergence of random variables. * Features applications of Law of Large Numbers. * Introduces Bernoulli and Poisson processes as well as discrete and continuous time Markov Chains with discrete states. * Includes illustrations and examples throughout, along with solutions to problems featured in this book. The authors present a unified and comprehensive overview of probability and Markov Chains aimed at educating engineers working with probability and statistics as well as advanced undergraduate students in sciences and engineering with a basic background in mathematical analysis and linear algebra.
(source: Nielsen Book Data)

• Preface; Program Committee;
• 1. Seneta;
• 2. Burrage et. al.;
• 3. Cloth and Haverkort;
• 4. Marie;
• 5. Latouche;
• 6. Spaey, Houdt and Blondia;
• 7. Vuuren and Adan;
• 8. Tian and Perros;
• 9. Dopper, Gaujal and Vincent;
• 10. Horvath and Telek;
• 11. Hilgers and Langville;
• 12. Da Silva and Rubino;
• 13. Mamoun, Busic, Fourneau and Pekergin;
• 14. Dayar et. al.;
• 15. Langville and Meyer;
• 16. Benzi and Ucar;
• 17. Tancrez and Semal;
• 18. Kirkland;
• 19. Dayar;
• 20. Sbeity and Plateau;
• 21. Kumar;
• 22. Caswell.

The Markov Anniversary Meeting, June 12-14, 2006 was the fifth in a series of conferences devoted to the Numerical Solution of Markov Chains (NSMC). The prior four NSMC conferences were held in 1990, 1995, 1999, and 2003. The first two conferences were held in Raleigh, North Carolina, the third moved overseas to Zarazoga, Spain, and the fourth was held in Urbana-Champaign, Illinois. This fifth and most recent Markov meeting occurred sooner than the usual 4-5 year schedule in order to celebrate two very important events: the 150th anniversary of Andrei A. Markov's birth and the 100th anniversar.

• I. Discrete Parameter
• § 1. Fundamental definitions
• § 2. Transition probabilities
• § 3. Classification of states
• § 4. Recurrence
• § 5. Criteria and examples
• § 6. The main limit theorem
• § 7. Various complements
• § 8. Repetitive pattern and renewal process
• § 9. Taboo probabilities
• § 10. The generating function
• § 11. The moments of first entrance time distributions
• § 12. A random walk example
• § 13. System theorems
• § 14. Functionals and associated random variables
• § 15. Ergodic theorems
• § 16. Further limit theorems
• § 17. Almost closed and sojourn sets
• II. Continuous Parameter
• § 1. Transition matrix: basic properties
• § 2. Standard transition matrix
• § 3. Differentiability
• § 4. Definitions and measure-theoretic foundations
• § 5. The sets of constancy
• § 6. Continuity properties of sample functions
• § 7. Further specifications of the process
• § 8. Optional random variable
• § 9. Strong Markov property
• § 10. Classification of states
• § 11. Taboo probability functions
• § 12. Last exit time
• § 13. Ratio limit theorems; discrete approximations
• § 14. Functionals
• § 15. Post-exit process
• § 16. Imbedded renewal process
• § 17. The two systems of differential equations
• § 18. The minimal solution
• § 19. The first infinity
• § 20. Examples.

From the reviews: J. Neveu, 1962 in Zentralblatt fr Mathematik, 92. Band Heft 2, p. 343: "Ce livre crit par l'un des plus minents spcialistes en la matire, est un expos trs dtaill de la thorie des processus de Markov dfinis sur un espace dnombrable d'tats et homognes dans le temps (chaines stationnaires de Markov)." N. Jain, 2008 in Selected Works of Kai Lai Chung, edited by Farid AitSahlia (University of Florida, USA), Elton Hsu (Northwestern University, USA), & Ruth Williams (University of California-San Diego, USA), Chapter 1, p. 15: "This monograph deals with countable state Markov chains in both discrete time (Part I) and continuous time (Part II). ... Much of Kai Lai's fundamental work in the field is included in this monograph. Here, for the first time, Kai Lai gave a systematic exposition of the subject which includes classification of states, ratio ergodic theorems, and limit theorems for functionals of the chain."

• Background : problem and methods
• One-dimensional calcium release
• Models of open and state blockers
• Properties of probability density functions
• Two-dimensional calcium release
• Computing theoretical drugs in the two-dimensional case
• Generalized systems governing probability density functions
• Calcium-induced calcium release
• Numerical drugs for calcium-induced calcium release
• A prototypical model of an ion channel
• Inactivated ion channels : extending the prototype model
• A simple model of the sodium channel
• Mutations affecting the mean open time
• The burst mode of the mutant sodium channel
• Action potentials : summing up the effect of loads of ion channels.

"Flow of ions through voltage gated channels can be represented theoretically using stochastic differential equations where the gating mechanism is represented by a Markov model. The flow through a channel can be manipulated using various drugs, and the effect of a given drug can be reflected by changing the Markov model. These lecture notes provide an accessible introduction to the mathematical methods needed to deal with these models. They emphasize the use of numerical methods and provide sufficient details for the reader to implement the models and thereby study the effect of various drugs. Examples in the text include stochastic calcium release from internal storage systems in cells, as well as stochastic models of the transmembrane potential. Well known Markov models are studied and a systematic approach to including the effect of mutations is presented. Lastly, the book shows how to derive the optimal properties of a theoretical model of a drug for a given mutation defined in terms of a Markov model."--Back cover

• Preface.- List of symbols.- Introduction.- Part I: Ergodic Theorems for Perturbed Alternating Regenerative Processes.- Ergodic Theorems for Perturbed Regenerative Processes.- Perturbed Alternating Regenerative Processes.- Ergodic Theorems for Regularly Perturbed Alternating Regenerative Processes.- Ergodic Theorems for Regularly Perturbed Alternating Regenerative Processes Compressed in Time.- Super-Long and Long Time Ergodic Theorems for Singularly Perturbed Alternating Regenerative Processes.- Short Time Ergodic Theorems for Singularly Perturbed Alternating Regenerative Processes.- Ergodic Theorems for Singularly Perturbed Alternating Regenerative Processes Compressed in Time.- Ergodic Theorems for Super-Singularly Perturbed Alternating Regenerative Processes.- Part II: Ergodic Theorems for Perturbed Multi-Alternating Regenerative Processes.- Perturbed Multi-Alternating Regenerative Processes.- Time-Space Aggregation of Regeneration Times for Perturbed Multi-Alternating Regenerative Processes.- Embedded Processes for Perturbed Multi-Alternating Regenerative Processes.- Ergodic Theorems for Perturbed Multi-Alternating Regenerative Processes.- Perturbed Renewal Equation.- Supplementary Asymptotic Results.- Methodological and Bibliography Notes.- References.- Index.- .
• (source: Nielsen Book Data)

This book is the second volume of a two-volume monograph devoted to the study of limit and ergodic theorems for regularly and singularly perturbed Markov chains, semi-Markov processes, and multi-alternating regenerative processes with semi-Markov modulation. The second volume presents a complete classification of ergodic theorems for alternating regenerative processes, including more than twenty-five such theorems. The text addresses new asymptotic recurrent algorithms of phase space reduction for multi-alternating regenerative processes modulating by regularly and singularly perturbed finite semi-Markov processes. It also features a new study of super-long, long, and short time ergodic theorems for these processes. The book also contains a comprehensive bibliography of major works in the field. It provides an effective reference for both graduate students as well as theoretical and applied researchers studying stochastic processes and their applications.
(source: Nielsen Book Data)

• Chapter 1: Basic Concepts
• Chapter 2: Introduction to Markov Processes
• Chapter 3: Discrete-Time Markov Chains
• Chapter 4: Continuous-Time Markov Chains
• Chapter 5: Markovian Queueing Systems
• Chapter 6: Markov Renewal Processes
• Chapter 7: Markovian Arrival Processes
• Chapter 8: Random Walk
• Chapter 9: Brownian Motion and Diffusion Processes
• Chapter 10: Controlled Markov Processes
• Chapter 11: Hidden Markov Models
• Chapter 12: Markov Point Processes.
• (source: Nielsen Book Data)

Markov processes are processes that have limited memory. In particular, their dependence on the past is only through the previous state. They are used to model the behavior of many systems including communications systems, transportation networks, image segmentation and analysis, biological systems and DNA sequence analysis, random atomic motion and diffusion in physics, social mobility, population studies, epidemiology, animal and insect migration, queueing systems, resource management, dams, financial engineering, actuarial science, and decision systems. Covering a wide range of areas of application of Markov processes, this second edition is revised to highlight the most important aspects as well as the most recent trends and applications of Markov processes. The author spent over 16 years in the industry before returning to academia, and he has applied many of the principles covered in this book in multiple research projects. Therefore, this is an applications-oriented book that also includes enough theory to provide a solid ground in the subject for the reader.
(source: Nielsen Book Data)

• 1. Introduction1.1 Thematic Context1.2 Capabilities of Markov Models1.3 Goal and Structure
• 2. Application Areas2.1 Speech2.2 Handwriting2.3 Biological Sequences2.4 Outlook Part I: Theory
• 3. Foundations of Mathematical Statistics3.1 Experiment, Event, and Probability3.2 Random Variables and Probability Distributions3.3 Parameters of Probability Distributions3.4 Normal Distributions and Mixture Density Models3.5 Stochastic Processes and Markov Chains3.6 Principles of Parameter Estimation3.7 Bibliographical Remarks
• 4. Vector Quantisation4.1 Definition4.2 Optimality4.3 Algorithms for Vector Quantiser Design (LLoyd, LBG, k-means)4.4 Estimation of Mixture Density Models4.5 Bibliographical Remarks
• 5. Hidden-Markov Models5.1 Definition5.2 Modeling of Output Distributions5.3 Use-Cases5.4 Notation5.5 Scoring (Forward algorithm)5.6 Decoding (Viterbi algorithm)5.7 Parameter Estimation (Forward-backward algorithm, Baum-Welch, Viterbi, and segmental k-means training)5.8 Model Variants5.9 Bibliographical Remarks
• 6. n-Gram Models6.1 Definition6.2 Use-Cases6.3 Notation6.4 Scoring6.5 Parameter Estimation (discounting, interpolation and backing-off)6.6 Model Variants (categorial models, long-distance dependencies)6.7 Bibliographical Remarks Part II: Practical Aspects
• 7. Computations with Probabilities7.1 Logarithmic Probability Representation7.2 Flooring of Probabilities7.3 Codebook Evaluation in Tied-Mixture Models7.4 Likelihood Ratios
• 8. Configuration of Hidden-Markov Models8.1 Model Topologies8.2 Sub-Model Units8.3 Compound Models8.4 Profile-HMMs8.5 Modelling of Output Probability Densities
• 9. Robust Parameter Estimation9.1 Optimization of Feature Representations (Principle component analysis, whitening, linear discriminant analysis)9.2 Tying (of model parameters, especially: mixture tying)9.3 Parameter Initialization
• 10. Efficient Model Evaluation10.1 Efficient Decoding of Mixture Densities10.2 Beam Search10.3 Efficient Parameter Estimation (forward-backward pruning, segmental Baum-Welch, training of model hierarchies)10.4 Tree-based Model Representations
• 11. Model Adaptation11.1 Foundations of Adaptation11.2 Adaptation of Hidden-Markov Models (Maximum-likelihood linear regression)11.3 Adaptation of n-Gram Models (cache models, dialog-step dependent models, topic-based language models)
• 12. Integrated Search12.1 HMM Networks12.2 Multi-pass Search Strategies12.3 Search-Space Copies (context and time-based tree copying strategies, language model look-ahead)12.4 Time-synchronous Integrated Decoding Part III: Putting it All Together
• 13. Speech Recognition13.1 Application-Specific Processing (feature extraction, vocal tract length normalization, ...)13.2 Systems (e.g. BBN Byblos, SPHINX III, ...)
• 14. Text Recognition14.1 Application-Specific Processing (linearization of data representation for off-line applications, preprocessing, normalization, feature extraction)14.2 Systems for On-line Handwriting Recognition14.3 Systems for Off-line Handwriting Recognition
• 15. Analysis of Biological Sequences15.1 Representation of Biological Sequences15.2 Systems (HMMer, SAM, Meta-MEME).
• (source: Nielsen Book Data)

Inequalities for polynomials and their derivatives are very important in many areas of mathematics, as well as in other computational and applied sciences; in particular they play a fundamental role in approximation theory. Here, not only Extremal Problems and Inequalities of Markov-Bernstein Type for Algebraic Polynomials, but also ones for trigonometric polynomials and related functions, are treated in an integrated and comprehensive style in different metrics, both on general classes of polynomials and on important restrictive classes of polynomials. Primarily for graduate and PhD students, this book is useful for any researchers exploring problems which require derivative estimates. It is particularly useful for those studying inverse problems in approximation theory.

• Zero-sum Markov games.- Discounted optimality criterion.- Average payoff criterion.- Empirical approximation-estimation algorithms in Markov games.- Difference-equation games: examples.- Elements from analysis.- Probability measures and weak convergence.- Stochastic kernels.- Review on density estimation. .
• (source: Nielsen Book Data)

This SpringerBrief deals with a class of discrete-time zero-sum Markov games with Borel state and action spaces, and possibly unbounded payoffs, under discounted and average criteria, whose state process evolves according to a stochastic difference equation. The corresponding disturbance process is an observable sequence of independent and identically distributed random variables with unknown distribution for both players. Unlike the standard case, the game is played over an infinite horizon evolving as follows. At each stage, once the players have observed the state of the game, and before choosing the actions, players 1 and 2 implement a statistical estimation process to obtain estimates of the unknown distribution. Then, independently, the players adapt their decisions to such estimators to select their actions and construct their strategies. This book presents a systematic analysis on recent developments in this kind of games. Specifically, the theoretical foundations on the procedures combining statistical estimation and control techniques for the construction of strategies of the players are introduced, with illustrative examples. In this sense, the book is an essential reference for theoretical and applied researchers in the fields of stochastic control and game theory, and their applications.
(source: Nielsen Book Data)

• Theorie elementaire des processus de markov
• Semi-groupes de feller
• Processus de hunt, processus standard
• Reduites, mesures harmoniques.

• Chaînes de Markov à temps discret
• Recuit simulé
• Gestions des approvisionnements
• Le processus de Galton-Watson
• Recherche de zones homogènes dans l'ADN
• Séquences exceptionnelles dans l'ADN
• Estimation du taux de mutation de l'ADN
• Chaînes de Markov à temps continu
• Files d'attente
• Éléments de fiabilité
• Lois de valeurs extrêmes
• Processus de coagulation et fragmentation.

Ce tome présente des modèles aléatoires élémentaires (chaînes de Markov à temps discret et continu, lois de valeurs extrêmes) et certaines de leurs applications courantes : algorithmes d'optimisation, gestion des approvisionnements, dimensionnement de files d'attente, fiabilité et dimensionnement d'ouvrages. Des problématiques plus récentes sont également abordées: recherche de séquences exceptionnelles et de zones homogènes de l'ADN, estimation du taux de mutation de l'ADN, phénomènes de coagulation de molécules de polymères ou d'aérosols. Ce tome s'adresse à un public très large d'étudiants et d'enseignants. Le prérequis pour sa lecture est la maîtrise du contenu d'un cours d'initiation aux probabilités.

• Preface.- Random Walks, a Good Place to Begin.- Doeblin's Theory for Markov Chains.- Stationary Probabilities.- More about the Ergodic Theory of Markov Chains.- Markov Processes in Continuous Time.- Reversible Markov Processes.- A minimal Introduction to Measure Theory.- Notation.- References.- Index.
• (source: Nielsen Book Data)

This book provides a rigorous but elementary introduction to the theory of Markov Processes on a countable state space. It should be accessible to students with a solid undergraduate background in mathematics, including students from engineering, economics, physics, and biology. Topics covered are: Doeblin's theory, general ergodic properties, and continuous time processes. Applications are dispersed throughout the book. In addition, a whole chapter is devoted to reversible processes and the use of their associated Dirichlet forms to estimate the rate of convergence to equilibrium. These results are then applied to the analysis of the Metropolis (a.k.a simulated annealing) algorithm. The corrected and enlarged 2nd edition contains a new chapter in which the author develops computational methods for Markov chains on a finite state space. Most intriguing is the section with a new technique for computing stationary measures, which is applied to derivations of Wilson's algorithm and Kirchoff's formula for spanning trees in a connected graph.
(source: Nielsen Book Data)