Holder Spaces

Let $\Omega \subset \mathbb{R}^n$ be an open set with $k \geq 0$ being an integer and $0 < \alpha \leq 1$. The space of Holder continuous functions $C^{0,\alpha}(\Omega)$ is defined as consisting of all those continuous functions $f \in C(\Omega)$ so that \[[f]_{C^{0,\alpha}(K)} := \sup_{x,y \in K, x\neq y} \left\{\frac{|u(x)-u(y)|}{|x-y|^\alpha}\right\} < \infty\] for every compact $K \subset \Omega$. Similarly, the space $C^{0,\alpha}(\bar{\Omega})$ is the set $f\in C(\bar{\Omega})$ so that $[f]_{C^{0,\alpha}(\bar{\Omega})} < \infty$. This space is equipped with the norm $\|f\|_{C^{0,\alpha}(\bar{\Omega})} := \|f\|_{C^0(\bar{\Omega})} + [f]_{C^{0,\alpha}(\bar{\Omega})}$.

$(C^{0,\alpha}(\bar{\Omega}), \|\cdot\|_{C^{0,\alpha}(\bar{\Omega})})$ is a Banach space

Let $f_n$ be a cauchy sequence in $C^{0,\alpha}(\bar{\Omega})$. Therefore, for every $\epsilon > 0$ there exits $M$ such that for all $n,m > M$, we have $\|f_n-f_m\|_{C^{0,\alpha}(\bar{\Omega})} < \epsilon$. This gives that $[f_n-f_m]_{C^{0,\alpha}(\bar{\Omega})} < \epsilon$ and $\|f_n -f_m\|_{C^0(\bar{\Omega})} < \epsilon$. From the latter we get that $f_n$ converges (in the sup norm) to a bounded continuous function $f$ (on $\bar{\Omega}$). From the former we get that $[f_n-f]_{C^{0,\alpha}(\bar{\Omega})} < \epsilon$ by letting $m \to \infty$. Therefore $\|f_n-f\|_{C^{0,\alpha}(\bar{\Omega})} \to 0$ as $n \to \infty$. It remains to show that $f \in {C^{0,\alpha}(\bar{\Omega})}$ but this follows easily by \[[f]_{C^{0,\alpha}(\bar{\Omega})} = [f-f_n+f_n]_{C^{0,\alpha}(\bar{\Omega})} \leq [f-f_n]_{C^{0,\alpha}(\bar{\Omega})} + [f_n]_{C^{0,\alpha}(\bar{\Omega})} < \infty\]

Note that this result does not depend on $\alpha$.

The space $C(\bar{\Omega})$

This post tries to establish that the above mentioned space is Banach. The proof is straight-forward but it is here for reference sake! Let $\Omega \subset \mathbb{R}^d$ be an open set. Denote by $C(\bar{\Omega})$ the set of all bounded continuous functions $u: \Omega \to \mathbb{R}$ which can be continuously extended to $\bar{\Omega}$ in a bounded manner. (These are exactly the same functions which are bounded and continuous on $\bar{\Omega}$) Now, we put the norm (called the sup norm) $\|f\| = \sup_{x \in \bar{\Omega}} f(x)$ on this space. Clearly, $C(\bar{\Omega})$ is a vector space and $\|f\| = \sup_{x \in \bar{\Omega}} |f(x)|$ defines a norm on $C(\bar{\Omega})$. The only non-trivial thing needed to prove this is that $\sup_x (|f(x)| + |g(x)|) \leq \sup_x |f(x)| + \sup_x |g(x)|$ (note that these two values need not be equal).

$C(\bar{\Omega})$ with the norm defined above is a Banach space

Let $f_n$ be a cauchy sequence in $C(\bar{\Omega})$, i.e. for every $\epsilon > 0$, there exists $N$ such that for all $n,m > N$, we have $\|f_n-f_m\| < \epsilon$, i.e. $\sup_x |f_n(x) - f_m(x)| < \epsilon$. This gives that $f_n(x)$ is a cauchy sequence for each $x\in \bar{\Omega}$. As $\mathbb{R}$ is complete, $f_n(x)$ converges for each $x$. Let the limit be $f(x)$ (i.e. $f(x) = \lim_{n \to \infty} f_n(x)$)

Note that we have convergence $f_n$ to $f$ in sup norm. (Just take the limit $m \to \infty$ in the cauchy criterion). It remains to show that $f$ is a bounded continuous function on $\bar{\Omega}$. \[\sup_x|f(x)| \leq \sup_x|f(x) - f_n(x)| + \sup_x|f_n(x)|\] For large enough $n$ the first term on the r.h.s can be made small and second term is finite as $f_n$ is bounded. Therefore $f$ is bounded.

As for continuity, we have \[|f(x)-f(y)| \leq |f(x)-f_n(x)| + |f_n(x)-f_m(x)| + |f_m(x)-f_m(y)| + |f_m(y)-f(y)|\] The first, second and the fourth terms can be made small by choosing large enough $n,m$ independent of $x,y$. The third term can be made arbitrarily small by choosing $y$ sufficiently close to $x$ due to the continuity of $f_m$.

Taylor's Theorem

The Taylor's theorem lets one to write down a function in terms of its derivatives and gives a direct way to compute limits in a few cases. For example, it is useful in computing limits of integrals as will be shown later in the post with the help of an example. Some of the basic theorems are stated/ proved first before giving a statement of the Taylor's theorem. The basic ideas are from Apostol. (All the integrals are Riemann).

Mean Value Theorem (Differentiation) : If $f:[a,b]\to \mathbb{R}$ is continuous on $[a,b]$ and differentiable on $(a,b)$ then $f(b)-f(a)=f^\prime(c)(b-a)$ for some $c\in (a,b)$.

Mean Value Theorem (Integration): Assume that $\int_a^b f(x)dx$ exists. Let $M = \sup_{x \in [a,b]} f(x), m = \inf_{x \in [a,b]} f(x)$. Then there exists a real number $c$ such that $m \leq c \leq M$ and \[\int_a^b f(x)dx = c(b-a)\] If $f$ is continuous on $[a,b]$ then, $c=f(x_0)$ for some $x_0\in [a,b]$

See Apostol for a Proof.

First Fundamental Theorem of Calculus: Assume that $\int_a^b f(x)dx$ exists. Let $F(x) = \int_a^x f(x)dx$. Then $F^\prime(x)$ exists at each point $x \in [a,b]$ where $f$ is continuous.

For $y\in (a,b)$ we have $F(y+h) - F(y) = \int_y^{y+h} f(x)dx = c(y,h)h$ where $\inf_{x \in [y,y+h]} f(x) \leq c(y,h) \leq \sup_{x \in [y,y+h]} f(x)$. Therefore, $\frac{F(y+h) - F(y)}{h} = c(y,h)$. If $f$ is continuous at $y$, then both $\inf_{x \in [y,y+h]} f(x)$ and $\sup_{x \in [y,y+h]} f(x)$ converge to $f(y)$ as $h \to 0$. The same reasoning will give that $\frac{F(y)-F(y-h)}{h}$ also converges to $f(y)$. Hence $F^\prime(y) = f(y)$. For the cases $y=a,y=b$, if we define one side limits of $F$, then clearly the above reasoning applies and we get $F^\prime(a)=f(a)$ and $F^\prime(b)=f(b)$ if $f$ is continuous at $a$ and $b$ respectively. (Ofcourse, continuity is also one-sided here).

Second Fundamental Theorem of Calculus: Assume that $\int_a^b f(x)dx$ exists. Let $g : [a,b] \to \mathbb{R}$ so that $g^\prime(x) = f(x)$ for all $(a,b)$ and $g(a) - g(a+) = g(b) - g(b-)$ (note that $g(a+),g(b-)$ should exist). Then \[\int_a^b f(x)dx = \int_a^b g^\prime(x)dx = g(b) - g(a)\]

Taylor's Theorm : Let $f:[a,b] \to \mathbb{R}$ be such that $f^{(n)}$ is finite on $(a,b)$ and $f^{(n-1)}$ is continuous on $[a,b]$. Assume that $c\in [a,b]$. Then for every $x \in [a,b], x\neq c$ there exits $x_1$ (which depends on $x$) interior to the interval joining $x$ and $c$ such that \[f(x) = f(c) + \sum_{k=1}^{n-1}\frac{f^{(k)}(c)}{k!}(x-c)^k + \frac{f^{(n)}(x_1)}{n!}(x-c)^n\]

If it is known that $f\in C^n[a,b]$ then Taylor's theorem gives a simple bounds as $f^{(n)}$ is bounded on $[a,b]$. A simple application of Taylor's theorem is given below. Suppose $f\in C^2_b(\mathbb{R})$ and we want to estimate the limit \[\lim_{t \to 0} \frac{1}{\sqrt{2\pi}}\int_{-\infty}^\infty \frac{f(x+y\sqrt{t})-f(x)}{t}\exp{(-\frac{1}{2}y^2)}dy\] Using Taylor's theorem, we can write $f(x+y\sqrt{t}) = f(x) + f^{(1)}(x)y\sqrt{t} +\frac{f^{(2)}(x+\theta(y)\sqrt{t})}{2}ty^2$. Therefore, the limit becomes \[\lim_{t \to 0} \frac{1}{\sqrt{2\pi}}\int_{-\infty}^\infty \frac{f^{(1)}(x)y\sqrt{t} +\frac{f^{(2)}(x+\theta(y)\sqrt{t})}{2}ty^2}{t}\exp{(-\frac{1}{2}y^2)}dy\] \[\lim_{t \to 0} \frac{1}{\sqrt{2\pi}}\int_{-\infty}^\infty \frac{f^{(2)}(x+\theta(y)\sqrt{t})}{2}y^2\exp{(-\frac{1}{2}y^2)}dy\] Now, using dominated convergence (based on the fact that $f^{(2)}(x)$ is bounded and continuous), the limit can be pushed inside the integral. Finally we get that the original expression equals \[\frac{1}{2}f^{(2)}(x)\]

$C^k$ Space

Let $\Omega \subset \mathbb{R}^d$ be an open set and $C^k(\bar{\Omega})$ be the set of all bounded functions $u : \Omega \to \mathbb{R}$ whose partial derivatives $D^a u$ for $0 \leq |a| \leq k$ are continuous on $\Omega$ and can be continuously extended to $\bar{\Omega}$ in a bounded way. (Here $D^a u$ represents partial derivative where $a=(a_1,a_2,\cdots,a_d)$ is a multi-index). Define a norm on $C^k(\bar{\Omega})$ by $\|u\|_{C^k} = \underset{0 \leq |a| \leq k}{\max} \underset{x \in \bar{\Omega}} \sup|D^au(x)|$. Then $(C^k(\bar{\Omega}),\|\cdot\|_{C^k})$ is a Banach Space.

Note that $\Omega$ can be unbounded here. The norm is still meaningful as the functions themselves are bounded. The following Mean-Value theorem in higher dimensions is used in the proof. Let $f : \Omega \to \mathbb{R}$ be differentiable (e.g. all partial derivatives of $f$ are continuous). Take two points $x,y\in \Omega$ and assume that the line joining the two points is also contained in $\Omega$. Define $g : [0,1] \to \mathbb{R}$ by $g(t) = f((1-t)x+ty)$. Now, clearly $g$ is differentiable and hence by the Mean Value Theorem, $g(1) - g(0) = g^\prime(c)$ where $c\in (0,1)$. Hence, $f(y) - f(x) = \langle \nabla f((1-c)x+cy),(y-x)\rangle$.

Assume now that $\{u_n\}$ is a cauchy-sequence and let $0 \leq |a| < k$. Also, let $\lim_{n\to \infty} u_n = v^0$. This limit exists and belongs to $C(\bar{\Omega})$. Similarly, let $\lim_{n\to \infty} D^au_n = v^a$. We need to show that $D^av^0 = v^a$.

As $\{u_n\}$ is cauchy, for a given $\epsilon > 0$, there exists $N$ such that $\underset{x \in \bar{\Omega}} \sup |\frac{\partial}{\partial x_i} D^a u_n (x) - \frac{\partial}{\partial x_i} D^a u_m (x)| < \epsilon$ for all $m,n \geq N$. Choose $f = D^au_n - D^au_m : \Omega \to \mathbb{R}$ in the previous paragraph. Then \begin{array} a\frac{\|(D^au_n(y) - D^au_m(y)) - (D^au_n(x) - D^au_m(x))\|}{\|y - x\|} &\leq \|\nabla f((1-c)x+cy)\| \\ &\leq \epsilon \sqrt{d} \end{array}

Now let $x\in \Omega$ and $\phi_n^i : \Omega_i \to \mathbb{R}$ given by $\phi_n^i(y) = \frac{\|D^au_n(x_1,x_2,\cdots,x_{i-1},y,x_{i+1},\cdots,x_d)- D^au_n(x_1,x_2,\cdots,x_{i-1},x_i,x_{i+1},\cdots,x_d)\|} {|y - x_i|}$. Here $\Omega_i \subset \mathbb{R}$ is an open ball around $x_i$ minus the point $x_i$ From the above we get that $\phi_n^i$ is uniformly-cauchy and hence converges uniformly in $\Omega_i$. Let $\lim_{n\to\infty} \phi_n^i(y)= \phi^i(y)$ in $\Omega_i$. Then, $\lim_{y \to x_i}\lim_{n \to \infty} \phi_n^i(y)= \lim_{n \to \infty} \lim_{y \to x_i} \phi_n^i(y)$ (i.e. the limits can be interchanged because of uniform convergence, see Rudin's PrinMathAnalysis Theorem 7.11) if $\lim_{y \to x_i} \phi_n^i(y)$ exists. In this case, this is true. Let $b=(a_1,a_2,\cdots,a_i+1,\cdots,a_n)$, then we get using recursion, \[D^bu(x)= \lim_{n\to \infty} D^{b}u_n(x)\]

Ascoli-Arzela Theorem

The content of this blog is almost a copy of the Wikipedia page on this topic. The proof given in Wikipedia is for real intervals. Here, a general compact set is considered. The sketch of the proof for general compact sets is already given in the Wikipedia page. This is merely a completion for the sake of reference.

Let $X$ be a compact space and $C(X;\mathbb{R}^n)$ be the set of continuous functions from $X$ to $\mathbb{R}^n$. Let $F \subset C(X;\mathbb{R}^n)$ be such that it satisfies the following two properties

• Equicontinuity : For each $\epsilon > 0$, for all $x\in X$, there exists a neighbourhood $U_x$ such that $\|f(y) - f(x)\| < \epsilon$ for all $y \in U_x$ and all $f \in F$
• Pointwise Boundedness : For each $x\in X$, $\sup\{\|f(x)\| : f \in F\} < \infty$

Then $\bar{F}$ is compact. (Here $C(X;\mathbb{R}^n)$ is given the metric $\|f\| = \sup\{\|f(x)\| : x \in X\}$

Let $\epsilon_n = \frac{1}{2^n}$. For each $x$, choose $U_x$ (from the equicontinuity of $\mathcal{F}$) such that the oscillation of any function in $\mathcal{F}$ is less than $\epsilon_1$. As $U_x$ form an open cover of $X$ and since $X$ is compact there exists a finite subcover which covers $X$. Denote this cover by $U_{x_{11}},U_{x_{12}},\cdots,U_{x_{1N_1}}$. With this process, we obtain for every $n$ a finite open cover $U_{x_{n1}},U_{x_{n2}},\cdots,U_{x_{nN_n}}$.

Rename the points $x_{11},x_{12},\cdots,x_{1N_1},x_{21}\cdots,x_{2N_2}\cdots$ as $x_1,x_2,\cdots$

Let $\{f_n\}$ be a sequence in $\mathcal{F}$. We want to show that there exists a sub-sequence which converges uniformly. As the space $\bar{\mathcal{F}}$ is a metric space (with the metric mentioned above) we get that it is compact.

Step 1 : As $\{\|f_n(x_1)\|\}$ is bounded, there exists a sub-sequence $\{f_{n_1}\}$ such that $f_{n_1}(x_1)$ converges. Now, we can choose a sub-sequence $f_{n_2}$ of $f_{n_1}$ such that $f_{n_2}(x_2)$ converges. This process is repeated ad-infinitum. Now, the "diagonal" sequence whose $m$th term is $m$th term in the $m$th subsequence $f_{n_m}$ is chosen and denoted by $f_m$. By construction, $f_m(x_i)$ converges for all $i$. This seems to be the central idea. For the next steps fix $l$.

Step 2: From the above, we know that for each $x_k$, there exists an integer $N(\epsilon,x_k)$ such that $\|f_n(x_k) - f_m(x_k)| < \epsilon_l$ for all $n,m > N(\epsilon_l,x_k)$.

Step 3 : Clearly, for $K=\sum_{i=1}^l N_i$ each open set $U_{lj}$, $1 \leq j \leq N_l$, contains at-least one point $x_k$ with $1 \leq k \leq K$

Step 4: For any $x \in X$, there exist $j,k$ such that $x \in U_{lj}, x_k \in U_{lj}$ where $1 \leq j \leq N_l$. For this $k$, \[\|f_n(x) - f_m(x)\| \leq \|f_n(x)-f_n(x_k)\| + \|f_n(x_k) - f_m(x_k)\| + \|f_m(x_k) - f_m(x)\| < 3\epsilon_l\] for all $n,m > \max(N(\epsilon,x_1),N(\epsilon,x_2),\cdots,N(\epsilon,x_K))$. Therefore, the sequence is uniformly-cauchy and hence converges to a continuous function $g \in C(X;\mathbb{R}^n)$. It is obvious that $g \in \bar{\mathcal{F}}$