path: root/final_project/paper/bnewbold_lch_report_draft.lyx

#LyX 1.4.3 created this file. For more info see http://www.lyx.org/
\lyxformat 245
\begin_document
\begin_header
\textclass article
\language english
\inputencoding auto
\fontscheme default
\graphics default
\paperfontsize default
\spacing single
\papersize default
\use_geometry true
\use_amsmath 1
\cite_engine basic
\use_bibtopic false
\paperorientation portrait
\rightmargin 3.5cm
\secnumdepth -1
\tocdepth 3
\paragraph_separation indent
\defskip medskip
\quotes_language english
\papercolumns 1
\papersides 1
\paperpagestyle default
\tracking_changes false
\output_changes false
\end_header

\begin_body

\begin_layout Title
Generic Operator Discovery
\end_layout

\begin_layout Date
\begin_inset Formula $\today$
\end_inset


\end_layout

\begin_layout Author
Laura Harris and Bryan Newbold for 6.945
\newline
{lch,bnewbold}@mit.edu
\end_layout

\begin_layout Abstract
We present a hardware design to convert captured images to an audio stream.
 There are a wealth of real time 
\emph on
software
\emph default
 implementations of the Fast Fourier Transform (FFT), but we use a Field
 Programable Gate Array 
\end_layout

\begin_layout Standard
\begin_inset LatexCommand \tableofcontents{}

\end_inset


\end_layout

\begin_layout Section
Overview
\end_layout

\begin_layout Standard
The Real-Time Audio Composition system allows for the conversion of visually
 encoded information into audio.
 A typical visual encoding program, such as Baudline, takes audio input
 and produces a scrolling visualization of the fast-Fourier transform (FFT).
 Our system could take one of these visualizations and produce ch serve
 as the visual spectrograph to interpreted and played back through headphones
 or speakers.
 A PS/2 mouse allows for control of the GUI system.
 Using the mouse, the user can interact with the buttons, the image itself,
 and a number of other features.
 An image of the GUI is shown in Figure 
\begin_inset LatexCommand \ref{fig:gui_photo}

\end_inset

.
\begin_inset Float figure
wide false
sideways false
status collapsed

\begin_layout Standard
\begin_inset Graphics
	filename gui_photo.jpg
	display none
	width 5in
	keepAspectRatio

\end_inset


\end_layout

\begin_layout Caption
A photograph of the GUI implemented in the Real Time Audio Composition system.
 Each area of the screen will be explained in detail.
\emph on

\newline
(Source: Team Member)
\emph default

\begin_inset LatexCommand \label{fig:gui_photo}

\end_inset


\end_layout

\end_inset


\end_layout

\begin_layout Standard
The GUI allows for a number of functions.
 An edit function, regrettably never fully implemented due to time constraints,
 would allow for the direct editing of captured images by manually selecting
 an eight bit binary lved with the time domain audio data to produce the
 audio signal.
 The AC97 audio chip in the FPGA runs at a clock speed of 48kHz so the audio
 samples must also be up-sampled to produce the final signal.
\end_layout

\begin_layout Standard
t vector with at least 1024 bins is required.
 Aliever, a full wrapper module was necessary to load samples in and out
 of the module, enable and disable processing, ensure sample index synchronizati
on, etc.
 The ifft_wrapper module ensures that only the 12-bit real components are
 written and that both the 8-bit phase and 12-bit complex components are
 tied to 0 at input and ignored at output.
\end_layout

\begin_layout Standard
\align center
\begin_inset Float table
placement h
wide false
sideways false
status collapsed

\begin_layout Standard
\align center
\begin_inset Tabular
<lyxtabular version="3" rows="8" columns="2">
<features>
<column alignment="center" valignment="top" width="0">
<column alignment="center" valignment="top" width="0">
<row bottomline="true">
<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
\begin_inset Text

\begin_layout Standard

\family roman
\series bold
\shape up
\size normal
\emph off
\bar no
\noun off
\color none
Setting
\end_layout

\end_inset
</cell>
<cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
\begin_inset Text

\begin_layout Standard

\family roman
\series bold
\shape up
\size normal
\emph off
\bar no
\noun off
\color none
Value
\end_layout

\end_inset
</cell>
</row>
<row topline="true">
<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
\begin_inset Text

\begin_layout Standard

\family roman
\series medium
\shape up
\size normal
\emph off
\bar no
\noun off
\color none
Generator Version
\end_layout

\end_inset
</cell>
<cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
\begin_inset Text

\begin_layout Standard

\family roman
\series medium
\shape up
\size normal
\emph off
\bar no
\noun off
\color none
 Fast Fourier Transform 3.2
\end_layout

\end_inset
</cell>
</row>
<row topline="true">
<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
\begin_inset Text

\begin_layout Standard

\family roman
\series medium
\shape up
\size normal
\emph off
\bar no
\noun off
\color none
Input Length
\end_layout

\end_inset
</cell>
<cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
\begin_inset Text

\begin_layout Standard

\family roman
\series medium
\shape up
\size normal
\emph off
\bar no
\noun off
\color none
2048 samples
\end_layout

\end_inset
</cell>
</row>
<row topline="true">
<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
\begin_inset Text

\begin_layout Standard

\family roman
\series medium
\shape up
\size normal
\emph off
\bar no
\noun off
\color none
Processing Mode
\end_layout

\end_inset
</cell>
<cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
\begin_inset Text

\begin_layout Standard

\family roman
\series medium
\shape up
\size normal
\emph off
\bar no
\noun off
\color none
Pipelined
\end_layout

\end_inset
</cell>
</row>
<row topline="true">
<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
\begin_inset Text

\begin_layout Standard

\family roman
\series medium
\shape up
\size normal
\emph off
\bar no
\noun off
\color none
Sample bitwidth
\end_layout

\end_inset
</cell>
<cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
\begin_inset Text

\begin_layout Standard

\family roman
\series medium
\shape up
\size normal
\emph off
\bar no
\noun off
\color none
12-bit signed
\end_layout

\end_inset
</cell>
</row>
<row topline="true">
<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
\begin_inset Text

\begin_layout Standard

\family roman
\series medium
\shape up
\size normal
\emph off
\bar no
\noun off
\color none
Output ordering
\end_layout

\end_inset
</cell>
<cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
\begin_inset Text

\begin_layout Standard

\family roman
\series medium
\shape up
\size normal
\emph off
\bar no
\noun off
\color none
Natural
\end_layout

\end_inset
</cell>
</row>
<row topline="true">
<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
\begin_inset Text

\begin_layout Standard

\family roman
\series medium
\shape up
\size normal
\emph off
\bar no
\noun off
\color none
Clock enable pin
\end_layout

\end_inset
</cell>
<cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
\begin_inset Text

\begin_layout Standard

\family roman
\series medium
\shape up
\size normal
\emph off
\bar no
\noun off
\color none
Enabled
\end_layout

\end_inset
</cell>
</row>
<row topline="true">
<cell alignment="center" valignment="top" topline="true" leftline="true" usebox="none">
\begin_inset Text

\begin_layout Standard

\family roman
\series medium
\shape up
\size normal
\emph off
\bar no
\noun off
\color none
Processing Stages
\end_layout

\end_inset
</cell>
<cell alignment="center" valignment="top" topline="true" leftline="true" rightline="true" usebox="none">
\begin_inset Text

\begin_layout Standard

\family roman
\series medium
\shape up
\size normal
\emph off
\bar no
\noun off
\color none
3 using BRAM
\end_layout

\end_inset
</cell>
</row>
</lyxtabular>

\end_inset


\end_layout

\begin_layout Caption
IFFT Auto-generation Settings
\begin_inset LatexCommand \label{tab:ifft_settings}

\end_inset


\end_layout

\end_inset


\end_layout

\begin_layout Standard
onversion is implemented by using each time-domain sample twice, then low
 pass filtering the output to remove the high frequency artifacts introduced.
 This low pass filter also conveniently filters out any noise or aliasing
 occurring in the frequencies above the 720 or so lowest frequencies actually
 specified by image data.
 Note that the up-sampling process also effectively halves the frequency
 corresponding to each bin in our frequency-domain spectral vectors, giving
 better frequency resolution in the lower, more human discernible regime.
\end_layout

\begin_layout Standard
The low pass filter is implemented almost exactly like in Lab 3, using 31
 tap filters generated in matlab using the command:
\end_layout

\begin_layout Verse

\family typewriter
round(fir1(30,.5)*(2^13))
\end_layout

\begin_layout Standard
Each tap is 14-bits signed, thus the scaling by 
\begin_inset Formula $2^{13}$
\end_inset

.
 The parameter 
\begin_inset Formula $W_{n}=0.5$
\end_inset

 specifies a cutoff frequency 
\end_layout

\begin_layout Standard
entire packet length is filled (128 values in our case), the IFFT is computed.
 Natural order refers to the ordering of bits on the output.
 Reverse ordering of bits is the natural mode of the IFFT, and since timing
 is not a huge concern at this stage, producing natural order bits is worth
 the time.
 The different stages of the IFFT input and output are controlled by an
 FSM as shown in Figure
\end_layout

\begin_layout Standard
Unless just reset, at which the FSM is immediately clock enabled and put
 in the setup state so the time domain taps can be initialized, the FSM
 naturally idles in the wait state.
 With the edge detection of a button press release within the bode plot,
 the FSM enters the setup state since it can be assumed that a value within
 the bode plot has changed.
 As soon as the ready for data flag is set high, the FSM enters the write
 state.
 The values from the bode array are fed into the IFFT, lagging behind three
 cycles as designated by the datasheet.
 After all data has been entered, the FSM enters the read state.
 The resulting tap indices and values are completely synchronous at this
 point and can be fed directly into the tap manager which will hold the
 resultant values.
\end_layout

\begin_layout Standard
Memory Management
\end_layout

\begin_layout Address

\emph on
Author: Dimitri Turbiner
\end_layout

\begin_layout Standard
The memory control module uses four interface modules to communicate with
 the Camera, GUI, and IFFT subsystems and two interface modules to control
 the physical SRAM chips on the LabKit.
 The camera_to_zbt interface module, keeps track of the sync signals coming
 from the Camera subsystem in order to compute the memory destination address.
 Incomming pixels are buffered and packed together into groups of four.
 When such a group of four is ready, the memory controller is signalled
 by camera_we.
 The input signals are all synchronized to the clock in the Camera subsystem
 -- 27mHz, while the output signals have to be synchronized at the memory
 clock -- 65mHz.
 Thus, the camera_to_zbt module has to perform clock resynchronization between
 the two subsystems it connects.
 The zbt_to_display, zbt_to_ifft, edit_to_zbt, modules all compute the memory
 destination address based on the requested pixel horizontal and vertical
 index (hcount and vcount), and when signalled by the memory controller
 that data is valid, pack or unpack four 8bit pixels to/from the 36 bit
 data bus.
 
\end_layout

\begin_layout Standard
The memory controller controls the two zbt ram access modules depending
 on which memory IO operations have been requested by the subsystems.
 The controller supports latching the various subsystem's data and address
 busses in order to guarantee minimum hold delays and to resolve simultaneous
 memory access requests.
 The latches control logic is built around a memory access priority hierarchy:
 
\end_layout

\begin_layout Quotation

\family typewriter
assign main_addr = camera_capture_request ? camera_addr :
\end_layout

\begin_layout Quotation

\family typewriter
\InsetSpace ~
\InsetSpace ~
\InsetSpace ~
\InsetSpace ~
\InsetSpace ~
\InsetSpace ~
\InsetSpace ~
\InsetSpace ~
\InsetSpace ~
\InsetSpace ~
\InsetSpace ~
\InsetSpace ~
\InsetSpace ~
\InsetSpace ~
\InsetSpace ~
\InsetSpace ~
\InsetSpace ~
\InsetSpace ~
\InsetSpace ~
(display_read_request ? display_addr :
\end_layout

\begin_layout Quotation

\family typewriter
\InsetSpace ~
\InsetSpace ~
\InsetSpace ~
\InsetSpace ~
\InsetSpace ~
\InsetSpace ~
\InsetSpace ~
\InsetSpace ~
\InsetSpace ~
\InsetSpace ~
\InsetSpace ~
\InsetSpace ~
\InsetSpace ~
\InsetSpace ~
\InsetSpace ~
\InsetSpace ~
\InsetSpace ~
\InsetSpace ~
\InsetSpace ~
(ifft_read_request ? ifft_addr :
\end_layout

\begin_layout Quotation

\family typewriter
\InsetSpace ~
\InsetSpace ~
\InsetSpace ~
\InsetSpace ~
\InsetSpace ~
\InsetSpace ~
\InsetSpace ~
\InsetSpace ~
\InsetSpace ~
\InsetSpace ~
\InsetSpace ~
\InsetSpace ~
\InsetSpace ~
\InsetSpace ~
\InsetSpace ~
\InsetSpace ~
\InsetSpace ~
\InsetSpace ~
\InsetSpace ~
(edit_write_request ? edit_addr :0))); 
\end_layout

\begin_layout Standard
problem was also noticed after this implementation.
 When the pixel itself was updated by multiplying the stored bode value
 by the incoming pixel data, a great deal of scattering across the boundaries
 of the image was noticed.
 By pipelining the data within the clocked region, this problem was removed
 and smooth fading occurred.
 (Note: the unimplemented edit mode code uses only slightly different math
 than the bode plot draw function, and should theoretically work, but was
 never tested).
\end_layout

\begin_layout Subsection
Audio Pipeline
\end_layout

\begin_layout Standard
A number of external tools were crucial for the development and debugging
 of the audio processing pipeline.
 An entire separate labkit module with a parameterized dummy sample generator
 replacing the camera and memory management and no GUI output.
 This allowed for debugging with consistent, simple, noise free spectral
 input.
 This labkit module was also wired to make full use of the hex display (to
 show bode tap index and both horizontal and vertical memory indexes) and
 the logic analyzer connections (all 4 16-bit connectors were used!).
 For instance, Figure shows the IFFT module output of a simple sine wave;
 the clock-like ticks of channels above and below the sine waveform show
 the request and ready signals of the various other stages of the audio
 pipeline.
 Note that this section of the spectral waveform shows the all zero padding
 from the windowing module; the single non-zero sample_value is off screen
 to the left, and the dummy ram module request/ready pins are not toggled.
\end_layout

\begin_layout Standard
A few other buttons and switches came in useful for debugging: Switch #7
 disables the regular audio output 
\end_layout

\begin_layout Standard
Conclusion
\end_layout

\begin_layout Address

\emph on
Author: Tyler Hutchison
\end_layout

\begin_layout Standard
The implementation of a real time audio composition system has been described.
 Using the GUI interface, coupled with complex memory interactions, and
 robust audio output, one can use visual information to encode 
\end_layout

\begin_layout Section
\start_of_appendix
Appendix: Code Listing
\begin_inset LatexCommand \label{sub:code}

\end_inset


\end_layout

\begin_layout Subsection
ghelper.scm
\end_layout

\begin_layout LyX-Code
////////////////////////////////////////////////////////////////////////////////
/
\end_layout

\begin_layout LyX-Code
//
\end_layout

\begin_layout LyX-Code
// 6.111 FPGA Labkit -- Template Toplevel Module
\end_layout

\begin_layout LyX-Code
//
\end_layout

\begin_layout LyX-Code
// Author: Nathan Ickes
\end_layout

\begin_layout LyX-Code
//
\end_layout

\begin_layout LyX-Code
// Edited: Nov 4, 2008, FTW
\end_layout

\begin_layout LyX-Code

\end_layout

\begin_layout LyX-Code

\end_layout

\begin_layout Subsection
discovery.scm
\end_layout

\begin_layout LyX-Code
//Dima Turbiner
\end_layout

\begin_layout LyX-Code
module zbt_to_ifft(input clk, reset, input [9:0]hcount, input [8:0]vcount,
 
\end_layout

\begin_layout LyX-Code
                                                input [35:0]data_main, input
 [35:0]data_overlay,
\end_layout

\begin_layout LyX-Code
                                                output reg [7:0]main_pixel,
 output reg [7:0]overlay_pixel, output [18:0]addr);
\end_layout

\begin_layout LyX-Code
   //**** CHECK THIS ADDRESSING
\end_layout

\begin_layout LyX-Code
   assign        addr = {2'b0, vcount, hcount[9:2]};
\end_layout

\begin_layout LyX-Code
   
\end_layout

\begin_layout LyX-Code
   wire [1:0]    hc4 = hcount[1:0];
\end_layout

\begin_layout LyX-Code

\end_layout

\end_body
\end_document