Simplify tree (#19)

* simplify the doc tree

* add faq and cookbook

* add cookbook files

* update build script

* fix contents.json file

* more examples, fix startBytes

* add FAQ

* Examples-> cookbook

* update diy dat with more details and examples

* fix package.json update

* cleanup Dat capitalization + typos, fix #17

* format ecosystem page

* add example dat links and typo fixes

2 files changed, 288 insertions, 0 deletions

diff --git a/docs/cookbook/browser.md b/docs/cookbook/browser.md
new file mode 100644
index 0000000..22010e4
--- /dev/null
+++ b/docs/cookbook/browser.md
@@ -0,0 +1,153 @@
+# Browser Dat
+
+Dat is written in JavaScript, so naturally, it can work entirely in the browser! The great part about this is that as more peers connect to each other in their client, the site assets will be shared between users rather hitting any sever.
+
+This approach is similar to that used in Feross' [Web Torrent](http://webtorrent.io). The difference is that Dat links can be rendered live and read dynamically, whereas BitTorrent links are static. In other words, the original owner can update a Dat and all peers will receive the updates automatically.
+
+OK, now for the goods:
+
+## Hyperdrive
+
+For now, there isn't an easy dat implementation for the browser. We have a simpler interface for node at [dat-js](http://github.com/joehand/dat-js).  
+
+If you want to get your hands dirty, here is the lower-level implementations to create a browser-based hyperdrive instance that will be compatible with dat.
+
+Hyperdrive will save the metadata (small) and the content (potentially large) separately. You can control where both of these are saved and how they are retrieved. These tweaks have huge impact on performance, stability, and user experience, so it's important to understand the tradeoffs.
+
+The first argument to `hyperdrive` will be the main database for all metadata and content. The `file` option can be supplied to specify how to read and write content data. If a `file` option is not supplied, the content will also be stored in the main database.
+
+```js
+var hyperdrive = require('hyperdrive')
+var drive = hyperdrive(<YOUR DATABASE HERE>, {file: <CONTENT DATABASE HERE>})
+```
+
+### The most basic example
+
+```js
+var hyperdrive = require('hyperdrive')
+var memdb = require('memdb')
+var swarm = require('hyperdrive-archive-swarm')
+
+var drive = hyperdrive(memdb())
+var archive = drive.createArchive()
+
+// joins the webrtc swarm
+swarm(archive)
+
+// this key can be used in another browser tab
+console.log(archive.key)
+```
+
+That's it. Now you are serving a dat-compatible hyperdrive from the browser. In another browser tab, you can connect to the swarm and download the data by using the same code as above. Just make sure to reference the hyperdrive you created before by using `archive.key` as the first argument:
+
+```js
+var drive = hyperdrive(memdb())
+var archive = drive.createArchive(<KEY HERE>)
+
+// joins the webrtc swarm
+swarm(archive)
+```
+
+For the full hyperdrive API and more examples, see the full [hyperdrive documentation](/hyperdrive).
+
+## Patterns for browser-based data storage and transfer
+
+There are a million different ways to store and retrieve data in the browser, and all have their pros and cons depending on the use case. We've compiled a variety of examples here to try to make it as clear as possible.
+
+### In-memory storage
+
+When the user refreshes their browser, they will lose all previous keys and data. The user will no longer be able to write more data into the hyperdrive.
+
+```js
+var hyperdrive = require('hyperdrive')
+var memdb = require('memdb')
+
+var drive = hyperdrive(memdb())
+var archive = drive.createArchive()
+```
+
+### Persistence with IndexedDB
+
+When the user refreshes their browser, their keys will be stored and retrieved.
+
+The best module to use for this is `level-browserify`:
+
+```js
+var hyperdrive = require('hyperdrive')
+var level = require('level-browserify')
+
+var drive = hyperdrive(level('./mydb'))
+var archive = drive.createArchive()
+```
+
+This will store all of the hyperdrive metadata *as well as content* in the client's IndexedDB. This is pretty inefficient. You'll notice that with this method that *IndexedDB will start to become full and the hyperdrive database will stop working as usual*.
+
+### Persistent metadata in IndexedDB with in-memory file content
+
+If you use level-browserify to store file content, you will quickly notice performance issues with large files. Writes after about 3.4GB will become blocked by the browser. You can avoid this by using in-memory storage for the file content.
+
+To do this, use [random-access-file-reader](https://github.com/mafintosh/random-access-file-reader) as the file writer and reader for the hyperdrive.
+
+```js
+var hyperdrive = require('hyperdrive')
+var level = require('level-browserify')
+var ram = require('random-access-memory')
+
+var drive = hyperdrive(level('./mydb'))
+var archive = drive.createArchive({
+  file: ram
+})
+```
+
+This works well for most cases until you want to write a file to hyperdrive that doesn't fit in memory.
+
+### Writing large files from the filesystem to the browser
+
+File writes are limited to the available memory on the machine. Files are buffered (read: copied) *into memory* while being written to the hyperdrive instance. This isn't ideal, but works as long as file sizes stay below system RAM limits.
+
+To fix this problem, you can use [random-access-file-reader](https://github.com/mafintosh/random-access-file-reader) to read the files directly from the filesystem instead of buffering them into memory.
+
+Here we will create a simple program that creates a file 'drag and drop' element on `document.body.` When the user drags files onto the element, pointers to them will be added to the `files` object.
+
+
+```js
+var drop = require('drag-drop')
+
+var files = {}
+
+drop(document.body, function (files) {
+  files[files[0].name] = files[0]
+})
+```
+
+Okay, that's pretty easy. Now let's add the hyperdrive. Hyperdrive needs to know what the pointers are, so when a peer asks for the file, it can read from the filesystem rather from memory. In other words, we are telling the hyperdrive which files it should index.
+
+```js
+var drop = require('drag-drop')
+var reader = require('random-access-file-reader')
+var hyperdrive = require('hyperdrive')
+var memdb = require('memdb')
+
+var files = {}
+
+var drive = hyperdrive(memdb())
+
+var archive = drive.createArchive({
+  file: function (name) {
+    return reader(files[name])
+  }
+})
+
+drop(document.body, function (files) {
+  files[files[0].name] = files[0]
+  // will index the file using hyperdrive without reading the entire file into ram
+  archive.append(files[0].name)
+})
+```
+
+## Unsatisfied?
+
+If you still aren't satisfied, come over to our community channels and ask a question. It's probably a good one and we should cover it in the documentation. Thanks for trying it out, and PRs always welcome!
+
+[![#dat IRC channel on freenode](https://img.shields.io/badge/irc%20channel-%23dat%20on%20freenode-blue.svg)](http://webchat.freenode.net/?channels=dat)
+[![datproject/discussions](https://badges.gitter.im/Join%20Chat.svg)](https://gitter.im/datproject/discussions?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
diff --git a/docs/cookbook/diy-dat.md b/docs/cookbook/diy-dat.md
new file mode 100644
index 0000000..5c3c21e
--- /dev/null
+++ b/docs/cookbook/diy-dat.md
@@ -0,0 +1,135 @@
+# Build with Dat
+
+In this guide, we will show how to develop applications with the Dat ecosystem. The Dat ecosystem is very modular making it easy to develop custom applications using Dat.
+
+For any Dat application, there are three essential modules you will start with: 
+
+1. [hyperdrive](https://npmjs.org/hyperdrive) for file synchronization and versioning
+2. [hyperdrive-archive-swarm](https://npmjs.org/hyperdrive-archive-swarm) helps discover and connect to peers over local networks and the internet
+3. A [LevelDB](https://npmjs.org/level) compatible database for storing metadata.
+
+The [Dat CLI](https://npmjs.org/dat) module itself combines these modules and wraps them in a command-line API. These modules can be swapped out for a similarly compatible module, such as switching LevelDb for [MemDB](https://github.com/juliangruber/memdb) (which we do in the first example). More details on how these module work together are available in [How Dat Works](how-dat-works.md).
+
+## Getting Started
+
+You will need node and npm installed to build with Dat. [Read more](https://github.com/maxogden/dat/blob/master/CONTRIBUTING.md#development-workflow) about our development work flow to learn how we manage our module dependencies during development.
+
+## Module #1: Download a File
+
+Our first module will download files from a Dat link entered by the user. View the code for this module on [Github](https://github.com/joehand/diy-dat-examples/tree/master/module-1).
+
+```bash
+mkdir module-1 && cd module-1
+npm init
+npm install --save hyperdrive memdb hyperdrive-archive-swarm
+touch index.js
+```
+
+For this example, we will use [memdb](https://github.com/juliangruber/memdb) for our database (keeping the metadata in memory rather than on the file system). In your `index.js` file, require the main modules and set them up:
+
+```js
+var memdb = require('memdb')
+var Hyperdrive = require('hyperdrive')
+var Swarm = require('hyperdrive-archive-swarm')
+
+var link = process.argv[2] // user inputs the dat link
+
+var db = memdb()
+var drive = Hyperdrive(db)
+var archive = drive.createArchive(link)
+var swarm = Swarm(archive)
+```
+
+Notice, the user will input the link for the second argument The easiest way to get a file from a hyperdrive archive is to make a read stream. `archive.createFileReadStream` accepts the index number of filename for the first argument. To display the file, we can create a file stream and pipe it to `process.stdout`.
+
+```js
+var stream = archive.createFileReadStream(0) // get the first file
+stream.pipe(process.stdout)
+```
+
+Now, you can run the module! To download the first file from our docs Dat, run:
+
+```
+node index.js 395e3467bb5b2fa083ee8a4a17a706c5574b740b5e1be6efd65754d4ab7328c2
+```
+
+You should see the first file in our docs repo.
+
+#### Module #1 Bonus: Display any file in the Dat
+
+With a few more lines of code, the user can enter a file to display from the Dat link.
+
+Challenge: create a module that will allow the user to input a Dat link and a filename: `node bonus.js <dat-link> <filename>`. The module will print out that file from the link, as we did above. To get a specific file you can change the file stream to use the filename instead of the index number:
+
+```js
+var stream = archive.createFileReadStream(fileName)
+```
+
+Once you are finished, see if you can view this file by running:
+
+```bash
+node bonus.js 395e3467bb5b2fa083ee8a4a17a706c5574b740b5e1be6efd65754d4ab7328c2 cookbook/diy-dat.md
+```
+
+[See how we coded it](https://github.com/joehand/diy-dat-examples/blob/master/module-1/bonus.js). 
+
+## Module #2: Download all files to computer
+
+This module will build on the last module. Instead of displaying a single file, we will download all of the files from a Dat into a local directory. View the code for this module on [Github](https://github.com/joehand/diy-dat-examples/tree/master/module-2).
+
+To download the files to the file system, instead of to a database, we will use the `file` option in `hyperdrive` and the [random-access-file](http://npmjs.org/random-access-file) module. We will also learn two new archive functions that make handling all the files a bit easier than the file stream in module #1. 
+
+Setup will be the same as before (make sure you install random-access-file and stream-each this time): 
+
+```bash
+mkdir module-2 && cd module-2
+npm init
+npm install --save hyperdrive memdb hyperdrive-archive-swarm random-access-file stream-each
+touch index.js
+```
+
+The first part of the module will look the same. We will add random-access-file (and [stream-each](http://npmjs.org/stream-each) to make things easier). The only difference is that we have to specify the `file` option when creating our archive:
+
+```js
+var memdb = require('memdb')
+var Hyperdrive = require('hyperdrive')
+var Swarm = require('hyperdrive-archive-swarm')
+var raf = require('random-access-file') // this is new!
+var each = require('stream-each')
+
+var link = process.argv[2]
+
+var db = memdb()
+var drive = Hyperdrive(db)
+var archive = drive.createArchive(link, {
+  file: function (name) {
+    return raf(path.join('download', name)) // download into a "download" dir
+  }
+})
+var swarm = Swarm(archive)
+```
+
+Now that we are setup, we can work with the archive. The `archive.download` function downloads the file content (to wherever you specified in the file option). To download all the files, we will need a list of files and then we will call download on each of them. `archive.list` will give us the list of the files. We use the stream-each module to make it easy to iterate over each item in the archive, then exit when the stream is finished.
+
+```js
+var stream = archive.list({live: false}) // Use {live: false} for now to make the stream easier to handle.
+each(stream, function (entry, next) {
+  archive.download(entry, function (err) {
+    if (err) return console.error(err)
+    console.log('downloaded', entry.name)
+    next()
+  })
+}, function () {
+  process.exit(0)
+})
+```
+
+You should be able to run the module and see all our docs files in the `download` folder:
+
+```bash
+node index.js 395e3467bb5b2fa083ee8a4a17a706c5574b740b5e1be6efd65754d4ab7328c2
+```
+
+## Module #3: Sharing a file
+
+## Module #4: Sharing a directory of files