Processing math: 21%

Doob's h-transform


doob_transform

Loosely speaking

Let A be a non-negative kernel on S and h that is A-harmonic positive function, then we call the transition kernel

Qh(x,y)=h(y)h(x)A(x,y),

the Doob h-transform of A.

Examples:

process h
Schur processes Schur functions
Whittaker processes Whittaker functions
q-Whittaker processes q-Whittaker functions
Macdonald processes Macdonald polynomials

In Markov doob integrability and the notes that depend on it we use this "loosely speaking" version of Doob's h-transform.

In the following we mostly follow exposition in Bloemendal's draft notes "Doob's h-transform: theory and examples".

Derivation

Consider a (continuous or discrete time) Markov process Xt on state S with kernel Pt. We associate with it a path space Ω with probability measures (Px)xS, and let Ft be the natural filtration of (Xt).

Let

I:={A:θ1tA=At0},

then any I-measurable function is shift-invariant: Hθt=Ht0, since

(Hθt)1B=θ1tH1(B).

Claim 1. There is a one-one correspondence between bounded I-measurable functions and bounded Pt-harmonic functions.

Proof. For any HI, let

h(x)=ExH

Then by the Markov property

h(Xt)=EXtH=Ex(Hθt|Ft)=Ex(H|Ft)

is a martingale, hence by Corollary 4 in infinitesimal_generator h is Pt-harmonic.

Conversely, given Pt-harmonic h, by martingale convergence theorem

H=lim

exists. And obviously H \in \mci. \square

L^\infty \mci is called the Poisson boundary.

For H = \ind_A for some A \in \mci, we define the conditional probability Q_x = P_x(\cdot | A) (see Claim 1 in radon_nikodym_derivative)

{d Q_x \over d P_x} = {\ind_A \over h(x)}.

By Claim 2 in radon_nikodym_derivative and Pf of Claim 1

{d Q_x \over d P_x}|_{\mcf_t} = {h(X_t) \over h(x)}

Let \tilde S = \{x \in S: h(x) > 0\} be accessible states.

Claim. Q_x is a probability measure on paths in \tilde S

Proof. \forall t, Q_x (h(X_t) > 0) = \expe_x \ind_{h (X_t) > 0} {h (X_t) \over h(x)} = \expe_x {h(X_t) \over h(x)} = 1, thus \forall t X_t \in \tilde S, P_x-a.s. \square

Define kernel Q_t

Q_t(x, d y) = {h(y) \over h(x)} P_t(x, dy) \qquad (1)

Claim. Q_t is a stochastic semigroup.

Proof.

\square

Theorem 1. Under Q_x with x \in \tilde S, X_t is Markov on \tilde S with kernel Q_t

Proof. Formally using Bayes theorem

P_x(X_{t + s} \in dy | A, \mcf_t) = {P_x(A | X_{t + s} = y, \mcf_t) P_x(X_{t + s} \in dy | \mcf_t) \over P_x(A | \mcf_t)} = {\expe_y(\ind_A \circ \theta_{t + s}) P_{X_t}(X_{t + s} \in dy) \over \expe_x (\ind_A | \mcf_t)} = {h(y) P_s(X_t, dy) \over h(X_t)}.

More precisely we want to show

\expe^Q_x (f(X_{t + s}) | \mcf_t) = h(X_t)^{-1} \expe_{X_t} f(X_s) h(X_s) \qquad Q_x \text{-a.s.}

We start from the RHS. For B \in \mcf_t,

\begin{align} \expe_x^Q &h(X_t)^{-1} \expe_{X_t} f(X_s) h(X_s) \ind_B \\ &= \expe^Q_x h(X_t)^{-1} \expe_x(f(X_{t + s}) h(X_{t + s}) | \mcf_t) \ind_B\\ &= \expe_x {h(X_t) \over h(x)} h(X_t)^{-1} \expe_x(f(X_{t + s}) h(X_{t + s}) | \mcf_t) \ind_B\\ &= \expe_x h(x)^{-1} f(X_{t + s}) h(X_{t + s}) \ind_B\\ &= \expe^Q_x f(X_{t + s}) \ind_B \end{align}

\square.

Continuous-time

Taking generator on both sides of (1) we have

L^Q = m_{1 / h} L^P m_h \qquad (1.5)

Claim. Specifically if X is a diffusion under P with SDE and generator

\begin{align} d X_t &= \sigma(X_t) d B_t + b(X_t) dt\\ L^P &= {1 \over 2} \sigma \sigma^T : \nabla \nabla^T + b \cdot \nabla = {1 \over 2} \sum a_{i j} \partial_{x_i x_j} + \sum b_i \partial_{x_i} (1.7) \end{align}

then

L^Q = L^P + a \nabla \log h \cdot \qquad (2)

Proof. plug L^P in (1.7) into (1.5), and noting L h = 0. \square

Remark. (2) can also be written as

L^Q f = L^P f + {1 \over h} \sigma^T \nabla h \cdot \sigma^T \nabla f

We also have that under Q,

d X_t = \sigma(X_t) d B_t + (b(X_t) + a \nabla h(X_t)) dt

namely a drift in the direction of increasing h is added.

Conditioning on a null event

When P_x(A) = 0 \forall x, one has to resort to the theory of Martin boundary (see perhaps e.g. [{sawyer97}]).

Heuristically, one may use a sequence A_n \in \mci (with harmonic function h_n) such that \bigcup A_n = A, and a sequence c_n \in \real such that \lim_n c_n h_n = h exists, is finite and positive.

Absorbing boundary

A special case is when we consider an absorbing boundary \partial S \subset S such that

P_t(x, dy) = \delta_x(dy), \forall x \in \partial S \forall t \ge 0

is the delta measure.

Let T = \tau_{\partial S}, the previous formula means X_t = X_{t \wedge T}.

We also consider Z \subset \partial S, and H = \{X_T \in Z\}.

Then the condition is "the Markov process hits Z when exitting S \setminus \partial S", i.e. \{X_T \in Z\}.

The harmonic function h is then a solution to the Dirichlet problem:

\begin{cases} L h(x) = 0, \qquad x \in S \setminus \partial S\\ h(x) = \ind_Z, \qquad x \in \partial S \end{cases}

The solution

$h(x) = P_x(X_T \in Z) = \expe_x h(X_T)

is a special case of potential theory in probability:

\begin{cases} L h(x) = 0, \qquad x \in S \setminus \partial S\\ h(x) = f(x), \qquad x \in \partial S. \end{cases}

has solution

u(x) = \expe_x f(X_T)

since u(X_t) is a martingale since

u(X_t) = \expe_{X_t} f(X_T) = \expe(f(X_{T + t}) | \mcf_t) = \expe(f(X_T) | \mcf_t)

due to the absorbing boundary.

Examples

Simple random walk on 0 : N hitting N before 0.

S = 0 : N, \partial S = \{0, N\}, Z = \{N\}.

If X_t is symmetric then the equation P h = h is

h(i) = {1 \over 2} (h(i + 1) + h(i - 1)), \qquad i = 1 : N - 1 \qquad (3)

combined with boundary condition it is a Dirichlet problem for Laplace equation:

\begin{cases} \Delta h(i) = 0, \qquad i = 1 : N - 1\\ h(0) = 0\\ h(N) = 1 \end{cases}

where \Delta is the discrete Laplacian.

Solving this equation gives

h(i) = i / N

and the kernel Q of the conditioned process by (1) is

Q(i, i + 1) = {i + 1 \over 2 i},\qquad Q(i, i - 1) = {i - 1 \over 2 i}, \qquad Q(i, \text{other j}) = 0 \qquad (4)

If X_t is asymmetric, with probability p of jumping from i to i + 1, then the equation (3) is now

h(x) = p h(x + 1) + (1 - p) h(x - 1)

with the same boundary condition. Solving it gives

h(i) = {1 - r^i \over 1 - r^N}

where r = {1 - p \over p}.

Brownian motion on [0, M] hitting M before 0

Same as in the RW example, but continuous time.

S = [0, M], \partial S = \{0, M\}, Z = \{M\}.

The equation L h = 0 with boundary condition is a Dirichlet problem

\begin{cases} {1 \over 2} h_{xx} (x) = 0 \qquad x \in (0, M)\\ h(0) = 0\\ h(M) = 1 \end{cases}

solving which gives

h(x) = x / M.

Hence by (2) we have

L^Q = {1 \over 2} \partial_{xx} + {1 \over x} \partial_x.

Brownian motion on [0, \pi] conditioned to remain in (0, \pi) up to time t_1

We use the space-time trick. S = [0, t_1] \times [0, \pi], \partial S = [0, t) \times \{0, \pi\} \cup \{t_1\} \times [0, \pi], Z = \{t_1\} \times [0, \pi].

Since the generator of (t, X_t) is \hat L = \partial_t + L, combining this with boundary condition we obtain the backward heat equation:

\begin{cases} \partial_t \hat h + {1 \over 2} \partial_{xx} \hat h = 0, \qquad (x, t) \in [0, t_1) \times (0, \pi) \\ h(t, 0) = h(t, \pi) = 0\qquad t \in [0, t_1)\\ h(t_1, x) = 1 \qquad x \in [0, \pi] \end{cases}

Solving this equation we have

\hat h(t, x) = \sum_{k \ge 1} c_k e^{- k^2 (t_1 - t)} \sin(k x)

where c_k = {4 \over k \pi} \ind_{2 \nmid k}.

So the new generator is

\hat L^Q = \partial_t + L + {\partial_{t, x} \hat h(x, t) \over \hat h(x, t)} \cdot \nabla_{t, x} = \partial_t + L + {\sum c_k k e^{-k^2(t_1 - t)} \cos(k x) \over \sum c_k e^{- k^2 (t_1 - t)} \sin(k x)} \partial_x + {\sum c_k e^{- k^2 (t_1 - t)} (- k^2) \sin k x \over \sum c_k e^{- k^2 (t_1 - t)} \sin k x} \partial_t.

Other examples include Bessel processes, dyson_brownian_motion, brownian_bridge, brownian_excursion, and various RS(K)-related models.

Discrete case (possibly Martin boundary)

This section follows exposition in [{oconnell03a}], let P be a substochastic transition matrix on a countably infinite state sapce S, and G to be the associated Green's function:

G(x, y) = \sum_{n \ge 0} P^n (x, y).

Assuming \exists x^* \in S s.t. 0 < G(x^*, y) < \infty \forall y \in S. Suppose h is a positive harmonic function w.r.t. P, then the Doob h-transform of P is the stochastic kernel P_h defined by

P_h(x, y) = {h(y) \over h(x)} P(x, y).

Theorem (Doob). If \exists f s.t.

\lim_{n \to \infty} {G(x, X(n)) \over G(x^*, X(n))} = f(x) \qquad a.s.

then h = C f for some C.

When conditioning on an event B with positive probability, we take

h(x) = P_x(B).

References