Completeness of orthonormal systems in $L^2$

[gugo82]

Usually the completeness of an orthonormal system [tex]$\mathcal{S} =\{ \varphi_n\}_{n\in \mathbb{N}}$[/tex] in [tex]$L^2([0,1])$[/tex] is proved by explicitly finding, for each [tex]$u\in L^2$[/tex], an approximating sequence [tex]$(u_n)\subset \text{span} \mathcal{S}$[/tex], or by showing that the only function which is othogonal to [tex]$\text{span} \mathcal{S}$[/tex] is the a.e. null one, or by verifying the Parseval identity again for each [tex]$u\in L^2$[/tex].
In all these cases one has to deal with formulas containing the "generic [tex]$L^2$[/tex] function" (like [tex]$\int_0^1 u(x)\ \phi_n(x)\ \text{d} x$[/tex], which contains [tex]$u$[/tex]), besides the elements of [tex]$\mathcal{S}$[/tex].

Proposition 3 to be demonstrated below reduces the proof of the completeness to the evaluation of the sum of a numerical series whose summands involves only the functions in [tex]$\mathcal{S}$[/tex]. Hence it turns out to be usefull to establish the completeness of the trigonometric or the Legendre polynomial system.

***

Few prerequisites:

Let [tex]$I:=[0,1]$[/tex] and [tex]$L^2(I):=\{ u:I\to \mathbb{R} :\ u \text{ is Lebesgue-measurable and } \int_0^1 u^2(x)\ \text{d} x <+\infty\}$[/tex] (the integral is taken in the Lebesgue sense).

The set [tex]$L^2(I)$[/tex] (up to the usual identification of functions which are equal a.e. in [tex]$I$[/tex]) can be turned into an Hilbert space with the usual pointwise sum and scalar product of functions and with the inner product defined by:

[tex]$\langle u,v\rangle :=\int_0^1 u(x)\ v(x)\ \text{d} x$[/tex];

the norm in [tex]$L^2(I)$[/tex] is obviously the one induced by [tex]$\langle \cdot ,\cdot \rangle$[/tex], i.e.:

[tex]$\lVert u\rVert_2 :=\left\{ \int_0^1 u(x)\ \text{d} x\right\}^{\frac{1}{2}}$[/tex].

The space so defined is separable, i.e. there exist systems of functions [tex]$\{ \varphi_n\}_{n\in \mathbb{N}}$[/tex] s.t. [tex]$L^2(I) =\overline{\text{span} \{ \varphi_n\}_{n\in \mathbb{N}}}$[/tex] (here and in what follows the bar denotes the closure operator w.r.t. the norm-induced topology).

We recall that a system of functions [tex]$\mathcal{S} :=\{ \varphi_n\}_{n\in \mathbb{N}}$[/tex] is said:

- linearly independent iff each finite subset [tex]$S\subset \mathcal{S}$[/tex] is linearly independent in the usual sense;

- orthonormal iff for [tex]$\varphi_n,\varphi_m \in \mathcal{S}$[/tex] is [tex]$\langle \varphi_n\varphi_m \rangle =\delta_{n,m}$[/tex] (where [tex]$\delta_{n,m}$[/tex] is the Kronecker delta);

- closed in [tex]$M\subseteq L^2(I)$[/tex] iff [tex]$\langle u,\varphi_n\rangle =0$[/tex] for all [tex]$n$[/tex]s implies [tex]$u=0$[/tex] a.e. in [tex]$[0,1]$[/tex];

- complete in [tex]$M\subseteq L^2(I)$[/tex] iff [tex]$M=\overline{\text{span} \mathcal{S}}$[/tex] (i.e. iff each function [tex]$u\in M$[/tex] can be approximated in norm with a sequence of linear combinations of the functions in [tex]$\mathcal{S}$[/tex]).

A remarkable result is the following:

If [tex]$\mathcal{S} :=\{ \varphi_n \}_{n\in \mathbb{N}} \subseteq L^2(I)$[/tex] is an orthonormal system and [tex]$M\subseteq L^2(I)$[/tex], then the following are equivalent:

i. [tex]$\mathcal{S}$[/tex] is closed in [tex]$M$[/tex];
ii. [tex]$\mathcal{S}$[/tex] is complete in [tex]$M$[/tex];
iii. for [tex]$u\in M$[/tex], [tex]$\lVert u\rVert_2^2 =\sum_{n=1}^{+\infty} (\langle u,\varphi_n\rangle )^2$[/tex];
iv. for [tex]$u,v\in M$[/tex], \( \langle u,v\rangle = \sum_{n=1}^{+\infty} \langle u,\phi_n\rangle \langle v,\phi_n\rangle \).

The equalities in iii and iv are usually said Parseval's identity and generalized Parseval's identity; Parseval identity implies that the Bessel's inequality:

[tex]$\sum_{n=1}^N(\langle u,\varphi_n \rangle )^2 \leq \lVert u\rVert_2^2$[/tex]

holds for each [tex]$u\in M$[/tex]. Indeed the classical Bessel's inequality holds in a more generale setting: in fact if [tex]$\mathcal{S} =\{ \varphi_n\}$[/tex] is an orthonormal system in [tex]$M$[/tex], then for each [tex]$u\in M$[/tex]:

[tex]$\sum_{n=1}^{+\infty} (\langle \varphi_n ,u\rangle)^2 \leq \lVert u\rVert_2^2$[/tex].

***

Problem:

Let [tex]$I:=[0,1]$[/tex].

1. If [tex]$(r_n)$[/tex] is an enumeration of [tex]$\mathbb{Q} \cap [0,1]$[/tex] (i.e. a bijection [tex]$\mathbb{N} \to \mathbb{Q} \cap [0,1]$[/tex]), then the system [tex]$\mathcal{T} :=\{ \psi_n\}_{n\in \mathbb{N}}$[/tex] containing the characteristic functions:

[tex]$\psi_n(x):=\chi_{[0,r_n[} (x) =\begin{cases} 1 &\text{, if } x\in [0,r_n[ \\ 0 &\text{, otherwise} \end{cases}$[/tex]

is linearly independent and closed, hence also complete in [tex]$L^2(I)$[/tex].

Hints: 1. The linear indipendence is trivial. For the closure of [tex]$\mathcal{T}$[/tex] just observe that if [tex]$u\in (\text{span} \mathcal{T} )^\bot$[/tex] then the integral function:

[tex]$U(x):=\int_0^x |u(t)|\ \text{d} t$[/tex]

is zero in all rational points.

2. Let [tex]$\mathcal{S}:=\{ \varphi_n \}_{n\in \mathbb{N}} \subset L^2(I)$[/tex] be an orthonormal system. Show that [tex]$\mathcal{S}$[/tex] is complete in [tex]$L^2(I)$[/tex] iff:

(*) [tex]$\forall x\in [0,1],\quad \sum_{n=1}^{+\infty} \left( \int_0^x \varphi_n(t)\ \text{d} t\right)^2 =x$[/tex].

Hints: 2. Realize that the equality in (*) can be regarded as the Parseval's identity for special [tex]$L^2(I)$[/tex] functions, hence necessity follows.
For the sufficiency, notice that (*) can be interpreted as a completeness condition for [tex]$\mathcal{S}$[/tex] in the [tex]$\text{span} \mathcal{T}$[/tex] and use density.

3. Let [tex]$\mathcal{S}$[/tex] be as in 2. Prove that [tex]$\mathcal{S}$[/tex] is complete in [tex]$L^2(I)$[/tex] iff:

(**) [tex]$\sum_{n=1}^{+\infty} \int_0^1 \left( \int_0^x \varphi_n (t)\ \text{d} t\right)^2\ \text{d} x =\frac{1}{2}$[/tex].

Hints: 3. Consider the deficit function [tex]$\Delta (x):=x-\sum_{n=1}^{+\infty} \left( \int_0^x \varphi_n(t)\ \text{d} t\right)^2$[/tex]: using the Bessel inequality, prove that [tex]$\Delta (x)\geq 0$[/tex], then prove that [tex]$x-\Delta (x)$[/tex] is continuous in [tex]$[0,1]$[/tex] (in particular, it is Hölder continuous with exponent [tex]$\tfrac{1}{2}$[/tex]).
Use equation (**) written in terms of [tex]$\Delta(x)$[/tex] and Dini's theorem on the uniform convergence of a series of positive function whose sum is continuous to prove necessity.
Finally, to prove sufficiency, apply Dini's theorem to (*).

4. How equations (*) and (**) have to be modified to hold if [tex]$I=[a,b]$[/tex]?

[gugo82]

"gugo82":3. Let [tex]$\mathcal{S}$[/tex] be as in 2. Prove that [tex]$\mathcal{S}$[/tex] is complete in [tex]$L^2(I)$[/tex] iff:

(**) [tex]$\sum_{n=1}^{+\infty} \int_0^1 \left( \int_0^x \varphi_n (t)\ \text{d} t\right)^2\ \text{d} x =\frac{1}{2}$[/tex].

Hints: 3. Consider the deficit function [tex]$\Delta (x):=x-\sum_{n=1}^{+\infty} \left( \int_0^x \varphi_n(t)\ \text{d} t\right)^2$[/tex]: using the Bessel inequality, prove that [tex]$\Delta (x)\geq 0$[/tex], then prove that [tex]$x-\Delta (x)$[/tex] is continuous in [tex]$[0,1]$[/tex] (in particular, it is Hölder continuous with exponent [tex]$\tfrac{1}{2}$[/tex]).
Use equation (**) written in terms of [tex]$\Delta(x)$[/tex] and Dini's theorem on the uniform convergence of a series of positive function whose sum is continuous to prove necessity.
Finally, to prove sufficiency, apply Dini theorem to (*)

[tex]$\Rightarrow)$[/tex] By 2 (*) holds.
The functions in the series at the LHS of (*) are non-negative, continuous (in fact absolutely continuous) and the sum in the RHS is also a continuous function; hence a classical Dini's theorem applies and the series [tex]\sum_{n=1}^{+\infty} \left( \int_0^x \varphi_n(t)\ \text{d} t\right)^2[/tex] converges uniformly in [tex]$I$[/tex].
Therefore a term by term integration of (*) over [tex]$I$[/tex] yields (**).

[tex]$\Leftarrow)$[/tex] By 2, in order to prove that [tex]$\mathcal{S}$[/tex] is complete in [tex]$L^2$[/tex] it suffices to show that (*) holds.
To this end, we put:

(1) [tex]$\Delta (x):=x-\sum_{n=1}^{+\infty} \left( \int_0^x \varphi_n(t)\ \text{d} t\right)^2$[/tex]

and observe that we have to prove [tex]$\Delta (x)=0$[/tex] in [tex]$I$[/tex] to get (*).

First, we observe that [tex]$\Delta (x)\geq 0$[/tex]: in fact, the classical Bessel's inequality applies to [tex]$\chi_{[0,x[} \in L^2$[/tex] yielding:

[tex]$\sum_{n=1}^{+\infty} \left( \int_0^x \varphi_n (t)\ \text{d} t\right)^2 =\sum_{n=1}^{+\infty} (\langle \varphi_n ,\chi_{[0,x[}\rangle )^2 \leq \lVert \chi_{[0,x[}\rVert_2^2 =x \quad \Rightarrow$[/tex]

[tex]$\Rightarrow \quad \Delta (x)=x-\sum_{n=1}^{+\infty} \left( \int_0^x \varphi_n (t)\ \text{d} t\right)^2 \geq 0$[/tex].

So, if it were possible to integrate term by term the LHS of (1), relation (**) could be written as:

[tex]$\int_0^1 \Delta (x)\ \text{d} x=0$[/tex];

notice that term by term integration is possible if, say, the series [tex]\sum_{n=1}^{+\infty} \left( \int_0^x \varphi_n(t)\ \text{d} t\right)^2[/tex] converges uniformly and a necessary condition to ensure uniform convergence of such a series is the continuity of [tex]$x-\Delta(x)$[/tex], because of the Dini's theorem. Therefore, in the hypotesis of continuity for [tex]$x-\Delta (x)$[/tex], the latter equality and the positiveness of [tex]$\Delta (x)$[/tex] would imply our claim.
Hence to conclude the proof it suffices to show that the function [tex]$x-\Delta(x) =\sum_{n=1}^{+\infty} \left( \int_0^x \varphi_n(t)\ \text{d} t\right)^2$[/tex] is continuous in [tex]$I$[/tex].

Choose [tex]$x,y \in I$[/tex] and consider the increment:

[tex]$ \delta (x,y):=\sum_{n=1}^{+\infty} \left( \int_0^x \varphi_n(t)\ \text{d} t\right)^2 - \left( \int_0^y \varphi_n(t)\ \text{d} t\right)^2$[/tex];

just to simplify notations we put [tex]$\alpha_n :=\int_0^x \varphi_n(t)\ \text{d} t$[/tex] and [tex]$\beta_n:= \int_0^y \varphi_n(t)\ \text{d} t$[/tex] so that:

[tex]$\delta (x,y):=\sum_{n=1}^{+\infty} \alpha_n^2 -\beta_n^2$[/tex].

Since [tex]$\alpha_n^2-\beta_n^2 =(\alpha_n +\beta_n)\ (\alpha_n-\beta_n)$[/tex], Cauchy-Schwarz's and convexity inequalities* applied to [tex]$\delta^2(x,y)$[/tex] yield:

(2) [tex]$\delta^2(x,y)\leq \sum_{n=1}^{+\infty} (\alpha_n+\beta_n)^2\ \cdot\ \sum_{n=1}^{+\infty} (\alpha_n-\beta_n)^2$[/tex]
[tex]$\leq 2\sum_{n=1}^{+\infty} \alpha_n^2+\beta_n^2\ \cdot \ \sum_{n=1}^{+\infty} (\alpha_n-\beta_n)^2$[/tex];

now, since [tex]$\alpha_n$[/tex]s and [tex]$\beta_n$[/tex]s are the Fourier coefficients of the functions [tex]$\chi_{[0,x[}$[/tex] and [tex]$\chi_{[0,y[}$[/tex] w.r.t. [tex]$\mathcal{S}$[/tex], Bessel's inequality applies and therefore:

[tex]$\sum_{n=1}^{+\infty} \alpha_n^2 \leq \lVert \chi_{[0,x[}\lVert_2^2\ =x \leq 1$[/tex],

[tex]$\sum_{n=1}^{+\infty} \beta_n^2 \leq \lVert \chi_{[0,y[}\lVert_2^2\ =y\leq 1$[/tex] and

[tex]$\sum_{n=1}^{+\infty} (\alpha_n -\beta_n)^2 \leq \lVert \chi_{[0,x[} -\chi_{[0,y[}\rVert_2^2\ =|x-y|$[/tex],

hence (2) implies:

[tex]$\delta^2 (x,y) \leq 4\ |x-y| \quad \Rightarrow$[/tex]

[tex]$\Rightarrow \quad |\delta (x,y)| \leq 2\ \sqrt{|x-y|}$[/tex].

The latter inequality shows that [tex]$x-\Delta (x)$[/tex] is an Hölder continuous function with exponent [tex]$\tfrac{1}{2}$[/tex].

As we said before, Dini's theorem applies and a term by term integration yields:

[tex]$\int_0^1 \Delta(x)\ \text{d} x =0$[/tex];

but [tex]$\Delta (x) \geq 0$[/tex] and [tex]$\Delta (x)=x-[x-\Delta (x)]$[/tex] is continuous in [tex]$I$[/tex], therefore [tex]$\Delta (x)=0$[/tex] as we claimed.

"gugo82":4. How equations (*) and (**) have to be modified to hold if [tex]$I=[a,b]$[/tex]?

A linear change of variable shows that (*) and (**) transform into:

(*) [tex]$\sum_{n=1}^{+\infty} \left( \int_a^x \varphi_n(t)\ \text{d} t\right)^2 =x-a$[/tex];

(**) [tex]$\sum_{n=1}^{+\infty} \int_a^b \left( \int_a^x \varphi_n(t)\ \text{d} t\right)^2 =\frac{1}{2}\ (b-a)^2$[/tex].


__________
* Let [tex]$(a_n),(b_n)$[/tex] be sequences of real numbers; then the following Cauchy-Schwarz's inequality holds:

[tex]$\left( \sum_{n=1}^{+\infty} |a_n|\ |b_n| \right)^2 \leq \sum_{n=1}^{+\infty} a_n^2\ \cdot\ \sum_{n=1}^{+\infty} b_n^2$[/tex].

Let [tex]$a,b \in \mathbb{R}$[/tex]; then the following (elementary) convexity inequality holds:

[tex]$(a+b)^2\leq 2\ (a^2+b^2)$[/tex].

[gugo82]

"gugo82":Let [tex]$I:=[0,1]$[/tex].

1. If [tex]$(r_n)$[/tex] is an enumeration of [tex]$\mathbb{Q} \cap [0,1]$[/tex] (i.e. a bijection [tex]$\mathbb{N} \to \mathbb{Q} \cap [0,1]$[/tex]), then the system [tex]$\mathcal{T} :=\{ \psi_n\}_{n\in \mathbb{N}}$[/tex] containing the characteristic functions:

[tex]$\psi_n(x):=\chi_{[0,r_n[} (x) =\begin{cases} 1 &\text{, if } x\in [0,r_n[ \\ 0 &\text{, otherwise}\end{cases}$[/tex]

is linearly independent and closed, hence also complete in [tex]$L^2(I)$[/tex].

Hints: 1. The linear indipendence is trivial. For the closure of [tex]$\mathcal{T}$[/tex] just observe that if [tex]$u\in (\text{span} \mathcal{T} )^\bot$[/tex] then the integral function:

[tex]$U(x):=\int_0^x |u(t)|\ \text{d} t$[/tex]

is zero in all rational points.

Assume [tex]$n\neq m$[/tex] and w.l.o.g. [tex]$r_n < r_m$[/tex]; if we put [tex]$s:=\alpha\ \psi_n +\beta\ \psi_m$[/tex] in [tex]$I$[/tex] then we have:

[tex]$s(x)=\beta$[/tex] for [tex]$x\in [r_n, r_m[$[/tex],

hence if [tex]$s=0$[/tex] in [tex]$L^2$[/tex] then [tex]$\beta =0$[/tex]; therefore also [tex]$\alpha =0$[/tex] (for [tex]$s(x)=\alpha +\beta$[/tex] in [tex]$x\in [0,r_n[$[/tex]).
The linear independence of [tex]$\mathcal{T}$[/tex] follows by induction.

Now let [tex]$u\in (\text{span} \mathcal{T})^\bot$[/tex]. Then for each [tex]$n\in \mathbb{N}$[/tex]:

[tex]$0= \langle |u|,\psi_n \rangle =\int_0^1 |u(x)|\ \psi_n(x)\ \text{d} x =\int_0^{r_n} |u(x)|\ \text{d} x$[/tex]

hence the function [tex]$U(x):=\int_0^x |u(t)|\ \text{d} t$[/tex] is zero on a dense subset (namely [tex]$\mathbb{Q} \cap I$[/tex]) in [tex]$I$[/tex]; but [tex]$U(x)$[/tex] is absolutely continuous in [tex]$I$[/tex], therefore [tex]$U$[/tex] is in fact nought in the whole of [tex]$I$[/tex]. This implies that [tex]$|u(x)|=0$[/tex] a.e. in [tex]$I$[/tex], that means [tex]$u=0$[/tex] in [tex]$L^2$[/tex].
The fact that [tex]$\mathcal{T}$[/tex] is closed in [tex]$L^2$[/tex] follows.

"gugo82":2. Let [tex]$\mathcal{S}:=\{ \varphi_n \}_{n\in \mathbb{N}} \subset L^2(I)$[/tex] be an orthonormal system. Show that [tex]$\mathcal{S}$[/tex] is complete in [tex]$L^2(I)$[/tex] iff:

(*) [tex]$\forall x\in [0,1],\quad \sum_{n=1}^{+\infty} \left( \int_0^x \varphi_n(t)\ \text{d} t\right)^2 =x$[/tex].

Hints: 2. Realize that the equality in (*) can be regarded as the Parseval identity for special [tex]$L^2(I)$[/tex] functions, hence necessity follows.
For the sufficiency, notice that (*) can be interpreted as a completeness condition for [tex]$\mathcal{S}$[/tex] in the [tex]$\text{span} \mathcal{T}$[/tex] and use density.

[tex]$\Rightarrow )$[/tex] Let [tex]$u(t):=\chi_{[0,x[} (t) \in L^2$[/tex] and suppose [tex]$\mathcal{S}$[/tex] complete in [tex]$L^2$[/tex]; then Parseval's identity iii applies to [tex]$u$[/tex] and yields:

[tex]$x=\lVert u\rVert_2^2 =\sum_{n=1}^{+\infty} (\langle \varphi_n ,u\rangle)^2 =\sum_{n=1}^{+\infty} \left( \int_0^x \varphi_n (t)\ \text{d} t \right)^2$[/tex]

which is (*).
[tex]$\Leftarrow )$[/tex] Conversely, suppose (*) holds. Then for each [tex]$\psi_m\in \mathcal{T}$[/tex] we have:

(a) [tex]$\sum_{n=1}^{+\infty} (\langle \varphi_n, \psi_m\rangle )^2= \sum_{n=1}^{+\infty} \left( \int_0^{r_m} \varphi_n(t)\ \text{d} t\right)^2 =r_m=\lVert \psi_m\rVert_2^2$[/tex],

therefore Parseval's identity w.r.t. [tex]$\mathcal{S}$[/tex] holds for each [tex]$\psi_m\in \mathcal{T}$[/tex].

Now notice that Parseval's identity w.r.t. an orthonormal system [tex]$\mathcal{U}$[/tex] holds for a function [tex]$f\in L^2$[/tex] iff [tex]$f$[/tex] can be approximated in norm by linear combinations of elements belonging to [tex]$\mathcal{U}$[/tex].
Therefore for each [tex]$\psi_m \in \mathbb{T}$[/tex] there exists at least a sequence [tex]$(\varphi_k^m) \in \text{span} \mathcal{S}$[/tex] s.t.:

[tex]$\lim_{k\to +\infty} \varphi_k^m = \psi_m$[/tex] in [tex]$L^2$[/tex];

if we choose [tex]$\alpha\ \psi_m +\beta\ \psi_l \in \text{span} \mathcal{T}$[/tex], then [tex]$\alpha\ \psi_m +\beta\ \psi_l$[/tex] will be approximated in norm by a sequence [tex]$(\alpha \varphi_k^m +\beta\ \varphi_k^l) \in \text{span} \mathcal{S}$[/tex], hence Parseval's identity w.r.t. [tex]$\mathcal{S}$[/tex] will hold for [tex]$\alpha\ \psi_m+\beta\ \psi_l$[/tex].
An induction argument shows that infact Parseval's identity w.r.t. [tex]$\mathcal{S}$[/tex] holds for each [tex]$u\in \text{span} \mathcal{T}$[/tex] and [tex]$\mathcal{S}$[/tex] is actually complete in [tex]$\text{span} \mathcal{T}$[/tex].

Finally let [tex]$u$[/tex] be any [tex]$L^2$[/tex] function and choose [tex]$\varepsilon >0$[/tex].
As we have seen in 1, [tex]$\mathcal{T}$[/tex] is closed in [tex]$L^2$[/tex] hence it is also complete; by definition there exists a function [tex]$\psi \in \text{span} \mathcal{T}$[/tex] s.t. [tex]$\lVert u-\psi \rVert_2<\tfrac{\varepsilon}{2}$[/tex].
Since [tex]$\mathcal{S}$[/tex] is closed in [tex]$\text{span} \mathcal{T}$[/tex], there exists a function [tex]$\varphi \in \text{span} \mathcal{S}$[/tex] s.t. [tex]$\lVert \psi -\varphi \lVert_2 <\tfrac{\varepsilon}{2}$[/tex]; hence by triangle inequality:

[tex]$\lVert u-\varphi \rVert_2<\varepsilon$[/tex].

Therefore [tex]$\mathcal{S}$[/tex] is complete in [tex]$L^2$[/tex].

[I think this was the hardest part of the problem to solve, because the equivalence between Parseval's inequality and approximation in norm is rarely used.]