# Lecture Notes
**[Josh's Notes for Lecture 2](/Lecture_2.pdf)**
#Why the Fourier decomposition is possible?
A Fourier series is a function of the form $\sum_{n=-\infty}^\infty a_n e^{inx}$ or $\sum_{n=-\infty}^\infty b_n \cos(n x) + c_n \sin (n x)$, depending on one's taste for the imaginary. The rumor on the street is that any periodic function (well, any nice one) can be expressed as a Fourier series: you hand me a function $f:[0,2\pi]\rightarrow \mathbb{C}$ and I hand you a list of real numbers $b_0,c_0,b_1,c_1,b_2,c_2,b_3,c_3,\dots$ such that
$f(x) = \sum_{n=-\infty}^\infty b_n \cos(n x) + c_n \sin (n x).$
If this exchange is always possible, there must be at least as many different lists of real numbers as there are nice periodic functions. So how many are there of each?
Infinity, of course. So to perform a meaningful check that the set of nice periodic functions is the same size as the set of lists of real numbers, we need a more refined notion of a size of a set than just giving its number of elements, or saying that it is infinite. We say that two sets have the same cardinality (our new word for size) if we can pair off their elements, giving a perfect matching between the sets.
## To show that $(0,1) \sim \mathbb R$
*--> could someone fill this out? *
## Cantor's proof for $\mathbb R > \mathbb N$
Suppose that we had a bijection between $\mathbb N$ and $\mathbb R$. Since $\mathbb R \sim \mathbb N$ In other words, we have a list of all the real numbers
1-->0.
## Proof that no. of available functions is greater than number of functions required to define the periodic function
Consider any arbitrary periodic function in the interval $[-\pi,\pi]$. This can be represented as a series of values at various points in the interval. For example,
$\qquad f(0) = ... , f(0.1) = ..., f(0.2) = ...$ and so on. At each point, we can assign any real number (i.e. $\in \mathbb R$). So, the number of possible periodic functions in an interval is of the order of $\mathbb R^{\mathbb R}$.
*--> don't quite remember how this goes.*
#Why Fourier decomposition is plausible?
To show that Fourier series is plausible, let us consider some arbitrary trignometric functions and see if it is possible to express them as the sum of sines and cosines:
$1.\quad\sin^2(x) = ?$
Based on the double angle formula,
$$\cos(2x) = 1 - 2 \sin^2(x)$$
Rearranging,
$$\sin^2(x) = \frac{1-\cos(2x)}{2}$$
$2.\quad\sin(2x)\cdot\cos(2x) = ?$
Based on the double angle formula,
$$\qquad\sin(2x) = 2\sin(x)\cos(x)$$
Rearranging,
$$\begin{array}{ccl}
\sin(2x)\cdot\cos(x) & = & [2\sin(x)\cos(x)]\cdot\cos(x)\\
& = & 2 \sin(x) [ 1 - \sin^2(x)]\\
& = & 2\sin(x) - 2\sin^3(x)\\
\end{array}$$
Based on the triple angle formula,
$$\sin(3x) = 3\sin(x) - 4\sin^3(x)$$
Rearranging,
$$\sin^3(x) = \frac{3\sin(x)-\sin(3x)}{4}$$
Substituting back in the former equation, we get
$$
\begin{array} {ccl}
\sin(2x) & = & 2\sin(x) - 2 [\frac{3\sin(x)-\sin(3x)}{4}]\\
& = & \frac{1}{2}\sin(x) + \frac{1}{2}\sin(3x)\\
\end{array}
$$
Thus, we see that both these functions could be expressed as sums of sines and cosines. It is possible to show that every product of trignometric functions can be expressed as a sum of sines and cosines:
$$
\begin{array}{ccl}
e^{i\theta} & = & \cos \theta + i \sin \theta\\
e^{-i\theta} & = & \cos \theta - i \sin \theta\\
\end{array}
$$
Solving for $\cos \theta$ and $\sin \theta$
$$
\begin{array}{ccl}
\cos \theta & = & \frac{1}{2}e^{i\theta} + \frac{1}{2}e^{-i\theta}\\
\sin \theta & = & \frac{1}{2i}e^{i\theta} - \frac{1}{2i}e^{-i\theta}\\
\end{array}
$$
It is easy to show that any product of cosines and sines can be expressed as the product of exponentials which will reduce to a sum of sines and cosines.
As a final test to see if the Fourier series really could exist for any periodic function, we consider a periodic function with a sharp peak as shown below
If it is possible to approximate the above function using a sum of sines and cosines, then it can be argued that *any* continuous periodic function can be expressed in a similar way. This is because any function could be expressed as a number of peaks at every position.
It turns out that the above function can be approximated as the difference of two cosines, namely, $\cos^{2n}(x) + cos^{2n+1}(x)$
 
Summing these two functions we get the following:
#Why does the Fourier decomposition actually work?
##Initial Hypothesis
Now, to prove that the Fourier series is indeed true, we begin with the following hypothesis:
Let $f : \mathbb I \rightarrow \mathbb C$ be a continuous, periodic function where $I$ is some time interval(period of the function). Then it can be expressed as :
$$
\begin{array}{ccl}
f & = & \Sigma a_n \, e^{inx}\\
& = & a_0 + \Sigma a_n\cos nx + \Sigma b_n\sin nx\\
\end{array}
$$
##Definition of Inner Product of Functions
We begin proving this hypothesis by considering that any function on the right-hand side of our hypothesis is uniquely defined by the co-efficients of the terms $a_0$ through $a_n$ and $b_1$ through $b_n$. This can be taken to mean that every function is really a vector in a $2n+1$-dimensional 'Hilbert space'.(*perhaps someone can clarify this?*)
We now proceed to define certain operations on these functions in Hilbert space. One operation that will be particularly useful is that of the inner product of two functions:
$$
\begin{array}{ccl}
inner product, (f,g) & = & \int_0^{2\pi} f \, g \,dx\\
\mid f \mid ^2 = (f, f) & = & \int_0^{2\pi} f^2 \,dx\\
\end{array}
$$
This is the inner product of 2 real-number functions. For a function on complex numbers, the above definition must be altered as follows:
$$
\begin{array}{ccl}
inner product, (f,g) & = & \int_0^{2\pi} f \,\bar g \,dx\\
\mid f \mid ^2 = (f, f) & = & \int_0^{2\pi} f^2 \,dx\\
\end{array}
$$
*Note: These are purely definitions, and we are defining the inner product to ensure that the inner product of f and f is a real number.*
##Basis Vectors of the Hilbert Space
The basis vectors of this Hilbert space are taken as follows:
basis vectors, $f_n = \frac{1}{\sqrt{2\pi}}e^{inx}$
Any basis vectors could conceivable have been assumed on the condition that the basis vectors are orthonormal. (*Note: These particular basis vectors are chosen to prove that Fourier series exists*)
In order to prove orthonormality of the basis vectors:
$$
\begin{array}{ccl}
(f_n,f_m) & = & \int_0^{2\pi} \, \frac{1}{\sqrt{2\pi}} \, e^{inx} \, \bar {\frac{1}{\sqrt{2\pi}} \, e^{inx}} \, dx\\
& = & \frac{1}{2\pi} \, \int_0^{2\pi} \, e^{i(n-m)x} \, dx \\
\end{array}
$$
The exponential can be expanded using $e^{ikx} = \cos kx + i \sin kx$. Then, integrating, we get the following:
$$
\begin{array}{ccl}
n = m \Rightarrow (f_n,f_m) & = & 1\\
n \neq m \Rightarrow (f_n,f_m) & = & 0\\
\end{array}
$$
which is the condition of orthonormality (the vectors are perpendicular and each has a length of 1 unit).
##Determining Coefficients of the Basis vectors
In any vector space, the inner product of a vector and its basis vector gives the coefficient. For example, consider a 2-dimensional vector as shown below:
The above vector $\vec v$ can be expressed in terms of the basis vectors $\vec e_1$ and $\vec e_2$ as follows:
$$
\begin{array}{ccl}
\vec v & = & a_1 \, \vec e_1 + a_2 \, \vec e_2\\
& = & a_1 \, \left( \begin{array}{c}1\\0\end{array} \right) + a_2 \, \left( \begin{array}{c}0\\1\end{array} \right)\\
& = & \left( \begin{array}{c}a_1\\a_2\end{array} \right) \\
\end{array}
$$
So,
$$
\begin{array}{ccl}
\vec v . \vec e_1 & = & a_1\\
\vec v . \vec e_2 & = & a_2\\
\end{array}
$$
Extending this principle to the case of an n-dimensional vector:
Let $f$ be the periodic function expressed as $f= \Sigma a_n \frac{1}{\sqrt{2\pi}} \, e^{inx} = \Sigma a_n \, f_n$ where $a_n \in \mathbb C$ and $f_n$ are the basis vectors.
Inner product of the vector (in this case the function $f$) with the some basis vector $f_m$ is:
$$
\begin{array}{ccl}
(f, f_m) & = & \left( \Sigma a_n\,f_n , f_m \right)\\
& = & \Sigma a_n\,\left(f_n , f_m \right)\\
\end{array}
$$
Due to orthonormality of basis vectors, the inner product in the right-hand side of the above equation is $0$ for all terms except $n = m$.
Thus,
$$ (f, f_m) = a_m $$
Using the definition of the inner product,
$$ a_m = \int_0^{2\pi} \, f \, \frac{1}{\sqrt{2\pi}} \, e^ {-inx} \, dx $$
This is the common definition for the terms of the Fourier series.
##Proving that this function is does indeed completely represent $f$
It is important to note at this point that we have simply expressed the periodic function $f$ in terms of a sum of arbitrary orthonormal vectors $f_n$. We haven't quite shown yet that the sum of orthonormal vectors actually completely represents $f$. Put in another way, there could be some components of $f$ that are not described by the vectors $f_n$. It is necessary to prove first that no other such components exist.
*--> don't quite remember this part*
Now, we know that the entire function space can be described by the defined basis vectors. We can now prove that, the sum of terms that we computed in the previous section does indeed converge to $f$ as follows:
*--> don't quite remember this part*
#Why the Fourier decomposition is useful?
Applications will be covered on Monday July 5, 2010.