aboutsummaryrefslogtreecommitdiffstats
path: root/proposals/0000-multiwriter.md
diff options
context:
space:
mode:
authorBryan Newbold <bnewbold@robocracy.org>2018-06-09 22:10:14 -0700
committerBryan Newbold <bnewbold@robocracy.org>2018-06-09 22:10:14 -0700
commit89c41746bc8c3a43651ede387b9ca39fc2ba2a2d (patch)
treed4e9a7b7d359feee2cd0cafd56c9e4bd03664707 /proposals/0000-multiwriter.md
parent33541cbfbcfc2d45fa5b4336b969fbd9181fb8ef (diff)
downloaddat-deps-89c41746bc8c3a43651ede387b9ca39fc2ba2a2d.tar.gz
dat-deps-89c41746bc8c3a43651ede387b9ca39fc2ba2a2d.zip
tweaks, updates, some progress
Diffstat (limited to 'proposals/0000-multiwriter.md')
-rw-r--r--proposals/0000-multiwriter.md65
1 files changed, 43 insertions, 22 deletions
diff --git a/proposals/0000-multiwriter.md b/proposals/0000-multiwriter.md
index fb60efc..b83606a 100644
--- a/proposals/0000-multiwriter.md
+++ b/proposals/0000-multiwriter.md
@@ -75,7 +75,10 @@ design and implement:
Before we go any further, a few definitions:
*Feed*: A hypercore feed: an append-only log of *Entries*, which can be
-arbitrary data blobs. Hyperdb is built on top of several Feeds.
+arbitrary data blobs.
+
+*Database*: in this context, a Hyperdb key/value database. Built from several
+Feeds (two Feeds per Writer).
*Writer*: a user (or user controlled device or software agent) that has a
distinct feed with a public/private key pair, and thus the ability to append
@@ -101,21 +104,33 @@ replicated data type" ([CRDT][crdt]).
[crdt]: https://en.wikipedia.org/wiki/Conflict-free_replicated_data_type
-## API
+## Core API
[api]: #api
-The `db.get(key)` method described in the [hyperdb DEP][dep-hyperdb] actually
-returns an array of values. If it is unambiguous what the consistent value is,
-the array will have only that one value. If there is a conflict, multiple
-values will be returned. If multiple values are returned, the writer should
-attempt to merge the values (or chose one over the other) and write the result
-to the database with `db.put(key, value)`.
+A "node" is a data structure representing a database entry, including the
+`key`, `value`, and feed that the entry is commited to.
+
+`db.get(key)` (as described in the [hyperdb DEP][dep-hyperdb])
+returns an array of nodes. If it is unambiguous what the consistent state of a key is,
+the array will have only that one value. If there is a conflict (ambiguity),
+multiple nodes will be returned.
+
+If multiple nodes are returned from a `get`, the writer should attempt to merge
+the values (or chose one over the other) and write the result to the database
+with `db.put(key, value)`. Client libraries can make this process more
+ergonomic by accepting a helper function to select between multiple nodes.
+Libraries can also offer an option to either directly return the value of a
+single node (instead of the node itself), or raise an error; this is likely to
+be more ergonomic for applications which do not intend to support multiple
+writers per database.
`db.authorize(key)` will write metadata to the local feed authorizing the new
feed (corresponding to `key`) to be included in the database.
-<!-- TODO: update link -->
-[dep-hyperdb]: https://github.com/bnewbold/dat-deps/blob/dep-hyperdb/proposals/0000-hyperdb.md
+`db.authorized(key)` (returning a boolean) indicates whether the given `key` is
+an authorized writer to the hyperdb database.
+
+[dep-hyperdb]: https://github.com/datprotocol/DEPs/blob/master/proposals/0004-hyperdb.md
## Scaling
@@ -131,7 +146,7 @@ scaling issues on their own. More real-world experience and benchmarking is
needed in this area.
-# Implementation
+# Implementation Details
[reference-documentation]: #reference-documentation
The complete protobuf schemas for the hyperdb "Entry" and "InflatedEntry"
@@ -142,9 +157,10 @@ message types (as specified in the hyperdb DEP) are:
message Entry {
required string key = 1;
optional bytes value = 2;
- required bytes trie = 3;
- repeated uint64 clock = 4;
- optional uint64 inflate = 5;
+ optional bool deleted = 3;
+ required bytes trie = 4;
+ repeated uint64 clock = 5;
+ optional uint64 inflate = 6;
}
message InflatedEntry {
@@ -154,11 +170,12 @@ message InflatedEntry {
required string key = 1;
optional bytes value = 2;
- required bytes trie = 3;
- repeated uint64 clock = 4;
- optional uint64 inflate = 5;
- repeated Feed feeds = 6;
- optional bytes contentFeed = 7;
+ optional bool deleted = 3;
+ required bytes trie = 4;
+ repeated uint64 clock = 5;
+ optional uint64 inflate = 6;
+ repeated Feed feeds = 7;
+ optional bytes contentFeed = 8;
}
```
@@ -173,9 +190,9 @@ The fields of interest for multi-writer are:
InflatedEntry, and only when feeds have changed.
When serialized on disk in a SLEEP directory, the original feed is written
-under `./source/`. If the "local" feed is different from the original feed, it
-is written to `./local/`. All other feeds are written under directories
-prefixed `./peers/<feed-discovery-key>/`.
+under `source/`. If the "local" feed is different from the original feed (aka,
+local is not the "Original Writer"), it is written to `local/`. All other feeds
+are written under directories prefixed `./peers/<feed-discovery-key>/`.
TODO: the above disk format is incorrect?
@@ -364,6 +381,10 @@ Answer: `get` always returns nodes (not just values), so context is included. In
What is a reasonable large number of writers to have in a single database?
Write "Scaling" section.
+The javascript hyperdb implementation has a new `deleted` flag and `Header`
+prefix message. These fall under the scope of the `hyperdb` DEP; should they be
+mentioned here?
+
# Changelog
[changelog]: #changelog