diff options
| -rw-r--r-- | rfcs/0000-hyperdb.md | 99 |
1 files changed, 99 insertions, 0 deletions
diff --git a/rfcs/0000-hyperdb.md b/rfcs/0000-hyperdb.md new file mode 100644 index 0000000..957cac2 --- /dev/null +++ b/rfcs/0000-hyperdb.md @@ -0,0 +1,99 @@ + +Status: Undefined (as of 2018-01-XXX) + +Type: Standard + +Github PR: (add HTTPS link here after PR is opened) + +Short Name: `0000-hyperdb` + +# Summary +[summary]: #summary + +HyperDB is an iteration on the hyperdrive directory tree implementation, +providing a general purpose key/value store on top of the hypercore append-only +log abstraction layer. Keys are path-like strings (eg, `/food/fruit/kiwi`), and +values are arbitrary binary blobs (with expected size of under a megabyte). +hyperdrive is expected to be re-implemented on top of HyperDB; the hyperdrive +API should be largely unchanged, but the metadata register format will be +backwards-incompatible. + +# Motivation +[motivation]: #motivation + +HyperDB is expected to drastically improve performance of dat clients when +working with archives containing tens of thousands of files in single +directories. This is a real-world bottleneck for several current users, with +basic local actions such as adding a directory taking an unacceptably long time +to complete. + +A secondary benefit is to refactor the tree-structured key/value API out of +hyperdrive, allowing third party code to build on this abstraction layer. + +# Usage Documentation +[usage-documentation]: #usage-documentation + +TODO: + +Document new features or processes as if this proposal has already been accepted and implemented. This section should introduce new concepts and terms, describe use cases or situations, and provide examples. If a user-facing feature is added, this should also include user-accessible documentation. + +# Reference Documentation +[reference-documentation]: #reference-documentation + +TODO: for now see [ARCHITECTURE.md][arch_md]. + +This section is a more formal description of the proposal, written as if it was a sub-section of the standard (for techincal proposals) or a formal process or "fine print" for process proposals. + +It should be unambiguous and cover all known corner-cases. Formal language (such as protobuf schemas or binary header diagrams) are appropriate here. + +# Drawbacks +[drawbacks]: #drawbacks + +TODO: + +Why should we *not* do this? + +# Rationale and alternatives +[alternatives]: #alternatives + +TODO: + +- Why is this design the best in the space of possible designs? +- What other designs have been considered and what is the rationale for not choosing them? +- What is the impact of not doing this? + +# Unresolved questions +[unresolved]: #unresolved-questions + +TODO: + +- What parts of the design do you expect to resolve through the RFC consensus process before this gets merged? +- What parts of the design do you expect to resolve through implementation and code review, or are left to independent library or application developers? +- What related issues do you consider out of scope for this RFC that could be addressed in the future independently of the solution that comes out of this RFC? + +# Migration logistics +[migration]: #migration + +HyperDB is not backwards compatible with the existing hyperdrive +implementation, meaning dat clients will need to support multiple on-disk +representations during a transition period. + +a new abstraction layer between hypercore (replicated append-only +logs) and hyperdrive (versioned file system abstraction). HyperDB provides an +efficient key/value database API, with path-like strings as keys and arbitrary +binary data (up to a reasonable chunk size) as values. HyperDB will require +breaking changes to dat clients, but will not require changes to the network +wire protocol. + +# Changelog +[changelog]: #changelog + +As of January 2018, @mafintosh is leading development of a hyperdb nodejs +module on [github](https://github.com/mafintosh/hyperdb), which is the basis +for this RFC. + +- 2017-12-06: @noffle publishes `ARCHITECTURE.md` overview in the + [hyperdb github repo][arch_md] +- 2018-01-XXX: First complete draft submitted for review + +[arch_md]: https://github.com/mafintosh/hyperdb/blob/master/ARCHITECTURE.md |