aboutsummaryrefslogtreecommitdiffstats
path: root/meta.dat.md
diff options
context:
space:
mode:
Diffstat (limited to 'meta.dat.md')
-rw-r--r--meta.dat.md101
1 files changed, 0 insertions, 101 deletions
diff --git a/meta.dat.md b/meta.dat.md
deleted file mode 100644
index 989879f..0000000
--- a/meta.dat.md
+++ /dev/null
@@ -1,101 +0,0 @@
-# meta.dat
-
-Dat uses a simple metadata file called `meta.dat`. The purpose of this file is to store the fingerprints of the files in a Dat repository. If you create a `meta.dat` file for a set of files, you can host it on a static HTTP server along with the files and Dat clients will be able to download and verify your files, even if you aren't running a Dat server!
-
-# File format
-
-```
-<Header><Entries Index...><Entries...>
-```
-
-The format is a header followed by an index of many entries. Entry order is based on the indexing determined by the [Flat In-Order Tree](hyperdrive.md#flat-in-order-trees) algorithm we use in Dat. After the entry index, a concatinated list of entries follows.
-
-### Header format
-
-```
-<varint header-length><header protobuf>
-```
-
-The header protobuf has this schema:
-
-``` protobuf
-message Header {
- required bytes datLink = 1;
- required uint64 entries = 2;
- optional bool isSigned = 3;
- optional string hashType = 4 [default = "sha256"];
- optional uint32 hashLength = 5 [default = 32];
- optional string signatureType = 6 [default = "ed25519"];
- optional uint32 signatureLength = 7 [default = 64];
-}
-```
-
-### Entry index format
-
-For non-signed entries:
-
-```
-<8-byte-chunk-end><chunk-hash>
-```
-
-The 8-byte-chunk-end is an unsigned big endian 64 bit integer that should be the absolute position in the file for the **end of the chunk**.
-
-For signed entries in live feeds (only applies to even numbered nodes e.g. leaf nodes):
-
-```
-<8-byte-chunk-end><chunk-signature><chunk-hash>
-```
-
-For any odd nodes, in either a live or a non-live feed, the non-signed entry format will be used.
-
-## Example
-
-Given a tree like this you might want to look up in a `meta.dat` file the metadata for a specific node:
-
-```
-0─┐
- 1─┐
-2─┘ │
- 3
-4─┐ │
- 5─┘
-6─┘
-```
-
-If you wanted to look up the metadata for 3, you could read the third (or any!) entry from meta.dat:
-
-First you have to read the varint at the beginning of the file so you know how big the header is:
-
-``` js
-var varint = require('varint') // https://github.com/chrisdickinson/varint
-var headerLength = varint.decode(firstChunkOfFile)
-```
-
-Now you can read the header from the file
-
-``` js
-var headerOffset = varint.encodingLength(headerLength)
-var headerEndOffset = headerOffset + headerLength
-var headerBytes = firstChunkOfFile.slice(headerOffset, headerEndOffset)
-```
-
-To decode the header use the protobuf schema. We can use the [protocol-buffers](https://github.com/mafintosh/protocol-buffers) module to do that.
-
-``` js
-var messages = require('protocol-buffers')(fs.readFileSync('meta.dat.proto'))
-var header = messages.Header.decode(headerBytes)
-```
-
-Now we have all the configuration required to calculate an entry offset.
-
-``` js
-var entryNumber = 42
-var entryOffset = headerEndOffset + entryNumber * (8 + header.hashLength)
-```
-
-If you have a signed feed, you have to take into account the extra space required for the signatures in the even nodes.
-
-``` js
-var entryOffset = headerLength + entryNumber * (8 + header.hashLength)
- + Math.floor(entryNumber / 2) * header.signatureLength
-```