path: root/Fourier Series.page

  
**[Josh's Notes for Lecture 2](/Lecture_2.pdf)**

#<b>Why the Fourier decomposition is possible?</b>
**[Josh's Notes for Lecture 2](/Lecture_2.pdf)**  
We first begin with a few basic identities on the size of sets. Then, we will show that the set of possible functions representing sets is not larger than the set of available functions. This at best indicates that the Fourier series is not altogether impossible.  
  
## To show that $(0,1) \sim \mathbb R$
--> could someone fill this out?  

## Cantor's proof for $\mathbb R > \mathbb N$
--> don't have the notes for this  

## Proof that no. of available functions is greater than number of functions required to define the periodic function
Consider any arbitrary periodic function in the interval $[-\pi,\pi]$. This can be represented as a series of values at various points in the interval. For example,  
$\qquad f(0) = ... , f(0.1) = ..., f(0.2) = ...$ and so on. At each point, we can assign any real number (i.e. $\in \mathbb R$). So, the number of possible periodic functions in an interval is of the order of $\mathbb R^{\mathbb R}$.
--> don't quite remember how this goes.

#<b>Why Fourier decomposition is plausible?</b>
To show that Fourier series is plausible, let us consider some arbitrary trignometric functions and see if it is possible to express them as the sum of sines and cosines:  

$1.\quad\sin^2(x) =  ?$  
   
Based on the double angle formula,  
  
$$\cos(2x) = 1 - 2 \sin^2(x)$$  
  
Rearranging,  
  
$$\sin^2(x) = \frac{1-\cos(2x)}{2}$$  
  
$2.\quad\sin(2x)\cdot\cos(2x) =  ?$  
   
Based on the double angle formula,  
  
$$\qquad\sin(2x) = 2\sin(x)\cos(x)$$  
  
Rearranging,
$$\begin{array}{ccl} 
\sin(2x)\cdot\cos(x) & = & [2\sin(x)\cos(x)]\cdot\cos(x)\\
 & = & 2 \sin(x) [ 1 - \sin^2(x)]\\
 & = & 2\sin(x) - 2\sin^3(x)\\
\end{array}$$
  

Based on the triple angle formula,  
  
$$\sin(3x) = 3\sin(x) - 4\sin^3(x)$$  
  
Rearranging,  
  
$$\sin^3(x) = \frac{3\sin(x)-\sin(3x)}{4}$$  

Substituting back in the former equation, we get  
  
$$
\begin{array} {ccl}
\sin(2x) & = & 2\sin(x) - 2 [\frac{3\sin(x)-\sin(3x)}{4}]\\ 
& = & \frac{1}{2}\sin(x) + \frac{1}{2}\sin(3x)\\
\end{array}
$$
  
Thus, we see that both these functions could be expressed as sums of sines and cosines. It is possible to show that every product of trignometric functions can be expressed as a sum of sines and cosines:  
  
$$
\begin{array}{ccl}
e^{i\theta} & = & \cos \theta + i \sin \theta\\
e^{-i\theta} & = & \cos \theta - i \sin \theta\\
\end{array}{ccl}
$$
  
Solving for $\cos \theta$ and $\sin \theta$  
  
$$
\begin{array}{ccl}
\cos \theta & = & \frac{1}{2}e^{i\theta} + \frac{1}{2}e^{-i\theta}\\
\sin \theta & = & \frac{1}{2i}e^{i\theta} - \frac{1}{2i}e^{-i\theta}\\
\end{array}
$$
  
It is easy to show that any product of cosines and sines can be expressed as the product of exponentials which will reduce to a sum of sines and cosines.  
  
As a final test to see if the Fourier series really could exist for any periodic function, we consider a periodic function with a sharp peak as shown below  
  
<center>![*Peak Function Image*](/peak_func.gif) </center> 
  
If it is possible to approximate the above function using a sum of sines and cosines, then it can be argued that *any* continuous periodic function can be expressed in a similar way. This is because any function could be expressed as a number of peaks at every position.  
    
It turns out that the above function can be approximated as the difference of two cosines, namely, $\cos^{2n}(x) + cos^{2n+1}(x)$  
<center>![$\cos^{2n}(x)$](/cos10x.gif) ![$cos^{2n+1}(x)$](/cos11x.gif)  </center>  
Summing these two functions we get the following:  
  
<center>![$\cos^{2n}(x) + cos^{2n+1}(x)$](/cos10x-cos11x.gif)</center>
  
#<b>Why does the Fourier decomposition actually work?</b>
##Initial Hypothesis
Now, to prove that the Fourier series is indeed true, we begin with the following hypothesis:  
Let $f : \mathbb I \rightarrow \mathbb C$ be a continuous, periodic function where  $I$ is some time interval(period of the function). Then it can be expressed as : 
  
$$
\begin{array}{ccl}
f & = & \Sigma a_n \, e^{inx}\\ 
& = & a_0 + \Sigma a_n\cos nx + \Sigma b_n\sin nx\\
\end{array}
$$  
  
##Definition of Inner Product of Functions  
We begin proving this hypothesis by considering that any function on the right-hand side of our hypothesis is uniquely defined by the co-efficients of the terms $a_0$ through $a_n$ and $b_1$ through $b_n$. This can be taken to mean that every function is really a vector in a $2n+1$-dimensional 'Hilbert space'.(*perhaps someone can clarify this?*)  
  
We now proceed to define certain operations on these functions in Hilbert space. One operation that will be particularly useful is that of the inner product of two functions:  
  
$$
\begin{array}{ccl}
inner product, (f,g) & = & \int_0^{2\pi} f \, g \,dx\\
\mid f \mid ^2  = (f, f) & = & \int_0^{2\pi} f^2 \,dx\\
\end{array}
$$

This is the inner product of 2 real-number functions. For a function on complex numbers, the above definition must be altered as follows:    
$$
\begin{array}{ccl}
inner product, (f,g) & = & \int_0^{2\pi} f \,\bar g \,dx\\
\mid f \mid ^2  = (f, f) & = & \int_0^{2\pi} f^2 \,dx\\
\end{array}
$$  
  
*Note: These are purely definitions, and we are  defining the inner product to ensure that the inner product of f and f is a real number.*

##Basis Vectors of the Hilbert Space  
The basis vectors of this Hilbert space are taken as follows:  
basis vectors, $f_n = \frac{1}{\sqrt{2\pi}}e^{inx}$  
  
Any basis vectors could conceivable have been assumed on the condition that the basis vectors are orthonormal. (*Note: These particular basis vectors are chosen to prove that Fourier series exists*)  
  
In order to prove orthonormality of the basis vectors:  
  
$$
\begin{array}{ccl}
(f_n,f_m) & = & \int_0^{2\pi} \, \frac{1}{\sqrt{2\pi}} \, e^{inx} \, \bar {\frac{1}{\sqrt{2\pi}} \, e^{inx}} \, dx\\
& = & \frac{1}{2\pi} \, \int_0^{2\pi} \, e^{i(n-m)x} \, dx \\
\end{array}
$$  
The exponential can be expanded using $e^{ikx} = \cos kx + i \sin kx$. Then, integrating, we get the following:    
$$
\begin{array}{ccl}
n = m \Rightarrow (f_n,f_m) & = & 1\\
n \neq m \Rightarrow (f_n,f_m) & = & 0\\
\end{array}
$$  
  
which is the condition of orthonormality (the vectors are perpendicular and each has a length of 1 unit).    
  
##Determining Coefficients of the Basis vectors
In any vector space, the inner product of a vector and its basis vector gives the coefficient. For example, consider a 2-dimensional vector as shown below:

<center>![Graph of a vector](/vector.gif)</center>

The above vector $\vec v$ can be expressed in terms of the basis vectors $\vec e_1$ and $\vec e_2$ as follows:
$$
\begin{array}{ccl}
\vec v & = & a_1 \, \vec e_1 + a_2 \, \vec e_2\\
& = &    a_1 \,   \left(  \begin{array}{c}1\\0\end{array} \right)  + a_2 \,  \left(  \begin{array}{c}0\\1\end{array} \right)\\
& = &  \left(  \begin{array}{c}a_1\\a_2\end{array} \right) \\
\end{array}
$$
  
So,  
$$
\begin{array}{ccl}
\vec v . \vec e_1 & = & a_1\\
\vec v . \vec e_2 & = & a_2\\
\end{array}
$$  
  
Extending this principle to the case of an n-dimensional vector:  
  
Let  $f$ be the periodic function expressed as $f= \Sigma a_n \frac{1}{\sqrt{2\pi}} \, e^{inx} = \Sigma a_n \, f_n$ where $a_n \in \mathbb C$ and $f_n$ are the basis vectors.  

Inner product of the vector (in this case the function $f$) with the some basis vector $f_m$ is:
$$
\begin{array}{ccl}
(f, f_m) & = & \left( \Sigma a_n\,f_n , f_m \right)\\
& = & \Sigma a_n\,\left(f_n , f_m \right)\\
\end{array}
$$  
  
Due to orthonormality of basis vectors, the inner product in the right-hand side of the above equation is $0$ for all terms except $n = m$.  
Thus,  
$$ (f, f_m) = a_m $$  
Using the definition of the inner product,  
$$ a_m = \int_0^{2\pi} \, f \, \frac{1}{\sqrt{2\pi}} \, e^ {-inx} \, dx $$  
  
This is the common definition for the terms of the Fourier series.

##Proving that this function is does indeed completely represent $f$
It is important to note at this point that we have simply expressed the periodic function $f$ in terms of a sum of arbitrary orthonormal vectors $f_n$. We haven't quite shown yet that the sum of orthonormal vectors actually completely represents $f$. Put in another way, there could be some components of $f$ that are not described by the vectors $f_n$. It is necessary to prove first that no other such components exist.
  
--> don't quite remember this part  
  
Now, we know that the entire function space can be described by the defined basis vectors. We can now prove that, the sum of terms that we computed in the previous section does indeed converge to $f$ as follows:
  
--> don't quite remember this part    
  
#<b>Why the Fourier decomposition is useful? </b>
Applications will be covered on Monday July 5, 2010. See you all soon!