Doob's h-transform
doob_transform
Loosely speaking
Let A be a non-negative kernel on S and h that is A-harmonic positive function, then we call the transition kernel
Qh(x,y)=h(y)h(x)A(x,y),the Doob h-transform of A.
Examples:
process | h |
---|---|
Schur processes | Schur functions |
Whittaker processes | Whittaker functions |
q-Whittaker processes | q-Whittaker functions |
Macdonald processes | Macdonald polynomials |
In Markov doob integrability and the notes that depend on it we use this "loosely speaking" version of Doob's h-transform.
In the following we mostly follow exposition in Bloemendal's draft notes "Doob's h-transform: theory and examples".
Derivation
Consider a (continuous or discrete time) Markov process Xt on state S with kernel Pt. We associate with it a path space Ω with probability measures (Px)x∈S, and let Ft be the natural filtration of (Xt).
Let
I:={A:θ−1tA=A∀t≥0},then any I-measurable function is shift-invariant: H∘θt=H∀t≥0, since
(H∘θt)−1B=θ−1tH−1(B).Claim 1. There is a one-one correspondence between bounded I-measurable functions and bounded Pt-harmonic functions.
Proof. For any H∈I, let
h(x)=ExHThen by the Markov property
h(Xt)=EXtH=Ex(H∘θt|Ft)=Ex(H|Ft)is a martingale, hence by Corollary 4 in infinitesimal_generator h is Pt-harmonic.
Conversely, given Pt-harmonic h, by martingale convergence theorem
H=limexists. And obviously H \in \mci. \square
L^\infty \mci is called the Poisson boundary.
For H = \ind_A for some A \in \mci, we define the conditional probability Q_x = P_x(\cdot | A) (see Claim 1 in radon_nikodym_derivative)
{d Q_x \over d P_x} = {\ind_A \over h(x)}.By Claim 2 in radon_nikodym_derivative and Pf of Claim 1
{d Q_x \over d P_x}|_{\mcf_t} = {h(X_t) \over h(x)}Let \tilde S = \{x \in S: h(x) > 0\} be accessible states.
Claim. Q_x is a probability measure on paths in \tilde S
Proof. \forall t, Q_x (h(X_t) > 0) = \expe_x \ind_{h (X_t) > 0} {h (X_t) \over h(x)} = \expe_x {h(X_t) \over h(x)} = 1, thus \forall t X_t \in \tilde S, P_x-a.s. \square
Define kernel Q_t
Q_t(x, d y) = {h(y) \over h(x)} P_t(x, dy) \qquad (1)Claim. Q_t is a stochastic semigroup.
Proof.
- Q_t is stochastic because h is P_t-harmonic.
- Q_t is a semigroup because it is a conjugate of P_t
\square
Theorem 1. Under Q_x with x \in \tilde S, X_t is Markov on \tilde S with kernel Q_t
Proof. Formally using Bayes theorem
P_x(X_{t + s} \in dy | A, \mcf_t) = {P_x(A | X_{t + s} = y, \mcf_t) P_x(X_{t + s} \in dy | \mcf_t) \over P_x(A | \mcf_t)} = {\expe_y(\ind_A \circ \theta_{t + s}) P_{X_t}(X_{t + s} \in dy) \over \expe_x (\ind_A | \mcf_t)} = {h(y) P_s(X_t, dy) \over h(X_t)}.More precisely we want to show
\expe^Q_x (f(X_{t + s}) | \mcf_t) = h(X_t)^{-1} \expe_{X_t} f(X_s) h(X_s) \qquad Q_x \text{-a.s.}We start from the RHS. For B \in \mcf_t,
\begin{align} \expe_x^Q &h(X_t)^{-1} \expe_{X_t} f(X_s) h(X_s) \ind_B \\ &= \expe^Q_x h(X_t)^{-1} \expe_x(f(X_{t + s}) h(X_{t + s}) | \mcf_t) \ind_B\\ &= \expe_x {h(X_t) \over h(x)} h(X_t)^{-1} \expe_x(f(X_{t + s}) h(X_{t + s}) | \mcf_t) \ind_B\\ &= \expe_x h(x)^{-1} f(X_{t + s}) h(X_{t + s}) \ind_B\\ &= \expe^Q_x f(X_{t + s}) \ind_B \end{align}\square.
Continuous-time
Taking generator on both sides of (1) we have
L^Q = m_{1 / h} L^P m_h \qquad (1.5)Claim. Specifically if X is a diffusion under P with SDE and generator
\begin{align} d X_t &= \sigma(X_t) d B_t + b(X_t) dt\\ L^P &= {1 \over 2} \sigma \sigma^T : \nabla \nabla^T + b \cdot \nabla = {1 \over 2} \sum a_{i j} \partial_{x_i x_j} + \sum b_i \partial_{x_i} (1.7) \end{align}then
L^Q = L^P + a \nabla \log h \cdot \qquad (2)Proof. plug L^P in (1.7) into (1.5), and noting L h = 0. \square
Remark. (2) can also be written as
L^Q f = L^P f + {1 \over h} \sigma^T \nabla h \cdot \sigma^T \nabla fWe also have that under Q,
d X_t = \sigma(X_t) d B_t + (b(X_t) + a \nabla h(X_t)) dtnamely a drift in the direction of increasing h is added.
Conditioning on a null event
When P_x(A) = 0 \forall x, one has to resort to the theory of Martin boundary (see perhaps e.g. [{sawyer97}]).
Heuristically, one may use a sequence A_n \in \mci (with harmonic function h_n) such that \bigcup A_n = A, and a sequence c_n \in \real such that \lim_n c_n h_n = h exists, is finite and positive.
Absorbing boundary
A special case is when we consider an absorbing boundary \partial S \subset S such that
P_t(x, dy) = \delta_x(dy), \forall x \in \partial S \forall t \ge 0is the delta measure.
Let T = \tau_{\partial S}, the previous formula means X_t = X_{t \wedge T}.
We also consider Z \subset \partial S, and H = \{X_T \in Z\}.
Then the condition is "the Markov process hits Z when exitting S \setminus \partial S", i.e. \{X_T \in Z\}.
The harmonic function h is then a solution to the Dirichlet problem:
\begin{cases} L h(x) = 0, \qquad x \in S \setminus \partial S\\ h(x) = \ind_Z, \qquad x \in \partial S \end{cases}The solution
$h(x) = P_x(X_T \in Z) = \expe_x h(X_T)is a special case of potential theory in probability:
\begin{cases} L h(x) = 0, \qquad x \in S \setminus \partial S\\ h(x) = f(x), \qquad x \in \partial S. \end{cases}has solution
u(x) = \expe_x f(X_T)since u(X_t) is a martingale since
u(X_t) = \expe_{X_t} f(X_T) = \expe(f(X_{T + t}) | \mcf_t) = \expe(f(X_T) | \mcf_t)due to the absorbing boundary.
Examples
Simple random walk on 0 : N hitting N before 0.
S = 0 : N, \partial S = \{0, N\}, Z = \{N\}.
If X_t is symmetric then the equation P h = h is
h(i) = {1 \over 2} (h(i + 1) + h(i - 1)), \qquad i = 1 : N - 1 \qquad (3)combined with boundary condition it is a Dirichlet problem for Laplace equation:
\begin{cases} \Delta h(i) = 0, \qquad i = 1 : N - 1\\ h(0) = 0\\ h(N) = 1 \end{cases}where \Delta is the discrete Laplacian.
Solving this equation gives
h(i) = i / Nand the kernel Q of the conditioned process by (1) is
Q(i, i + 1) = {i + 1 \over 2 i},\qquad Q(i, i - 1) = {i - 1 \over 2 i}, \qquad Q(i, \text{other j}) = 0 \qquad (4)If X_t is asymmetric, with probability p of jumping from i to i + 1, then the equation (3) is now
h(x) = p h(x + 1) + (1 - p) h(x - 1)with the same boundary condition. Solving it gives
h(i) = {1 - r^i \over 1 - r^N}where r = {1 - p \over p}.
Brownian motion on [0, M] hitting M before 0
Same as in the RW example, but continuous time.
S = [0, M], \partial S = \{0, M\}, Z = \{M\}.
The equation L h = 0 with boundary condition is a Dirichlet problem
\begin{cases} {1 \over 2} h_{xx} (x) = 0 \qquad x \in (0, M)\\ h(0) = 0\\ h(M) = 1 \end{cases}solving which gives
h(x) = x / M.Hence by (2) we have
L^Q = {1 \over 2} \partial_{xx} + {1 \over x} \partial_x.Brownian motion on [0, \pi] conditioned to remain in (0, \pi) up to time t_1
We use the space-time trick. S = [0, t_1] \times [0, \pi], \partial S = [0, t) \times \{0, \pi\} \cup \{t_1\} \times [0, \pi], Z = \{t_1\} \times [0, \pi].
Since the generator of (t, X_t) is \hat L = \partial_t + L, combining this with boundary condition we obtain the backward heat equation:
\begin{cases} \partial_t \hat h + {1 \over 2} \partial_{xx} \hat h = 0, \qquad (x, t) \in [0, t_1) \times (0, \pi) \\ h(t, 0) = h(t, \pi) = 0\qquad t \in [0, t_1)\\ h(t_1, x) = 1 \qquad x \in [0, \pi] \end{cases}Solving this equation we have
\hat h(t, x) = \sum_{k \ge 1} c_k e^{- k^2 (t_1 - t)} \sin(k x)where c_k = {4 \over k \pi} \ind_{2 \nmid k}.
So the new generator is
\hat L^Q = \partial_t + L + {\partial_{t, x} \hat h(x, t) \over \hat h(x, t)} \cdot \nabla_{t, x} = \partial_t + L + {\sum c_k k e^{-k^2(t_1 - t)} \cos(k x) \over \sum c_k e^{- k^2 (t_1 - t)} \sin(k x)} \partial_x + {\sum c_k e^{- k^2 (t_1 - t)} (- k^2) \sin k x \over \sum c_k e^{- k^2 (t_1 - t)} \sin k x} \partial_t.Other examples include Bessel processes, dyson_brownian_motion, brownian_bridge, brownian_excursion, and various RS(K)-related models.
Discrete case (possibly Martin boundary)
This section follows exposition in [{oconnell03a}], let P be a substochastic transition matrix on a countably infinite state sapce S, and G to be the associated Green's function:
G(x, y) = \sum_{n \ge 0} P^n (x, y).Assuming \exists x^* \in S s.t. 0 < G(x^*, y) < \infty \forall y \in S. Suppose h is a positive harmonic function w.r.t. P, then the Doob h-transform of P is the stochastic kernel P_h defined by
P_h(x, y) = {h(y) \over h(x)} P(x, y).Theorem (Doob). If \exists f s.t.
\lim_{n \to \infty} {G(x, X(n)) \over G(x^*, X(n))} = f(x) \qquad a.s.then h = C f for some C.
When conditioning on an event B with positive probability, we take
h(x) = P_x(B).References
- [oconnell03a] Conditioned random walks and the RSK correspondence, , Journal of Physics A: Mathematical and General, Vol. 36, No. 12, p.3049–3066, March 2003.
- [sawyer97] Martin boundaries and random walks, , Contemporary Mathematics, Vol. 206, p.17–44 1997.