From d542e4e12df6031139f5ff9c8cc607b8524cf948 Mon Sep 17 00:00:00 2001 From: Bryan Newbold Date: Tue, 5 Dec 2017 23:51:06 -0800 Subject: update README/TODO --- README.md | 9 ++++++--- TODO | 33 ++++++++++++++------------------- 2 files changed, 20 insertions(+), 22 deletions(-) diff --git a/README.md b/README.md index f6a06fc..e6a5e46 100644 --- a/README.md +++ b/README.md @@ -22,6 +22,7 @@ it will eat-your-data! - [x] read data entries by index - [x] append data entries - [x] verify entire register (signatures and merkel tree) + - [ ] receive and insert data out of order - [ ] bitfields - [ ] Drive metadata and files - [x] read full history ("log") @@ -34,7 +35,10 @@ it will eat-your-data! - [ ] receive entire register from a known host - [ ] share (upload) register to a known host - [ ] bitfields - - [ ] lookup hosts in DHT swarm by discovery key +- [ ] Discovery + - [x] centralized DNS + - [ ] mDNS (local DNS) + - [ ] DHT (distributed hash table) - [ ] Wrapper commands - [ ] clone - [ ] share @@ -44,7 +48,7 @@ it will eat-your-data! ### Differences from dat -A few simplifications were made compared to the regular dat client: +Significant simplifications were made compared to the regular dat client: - Content data is always stored in the SLEEP directory (`content.data`), instead of just having the latest data in files in the working directory. @@ -54,7 +58,6 @@ A few simplifications were made compared to the regular dat client: - Tracking of remote node state (bitfields) is minimized as much as possible. - Almost everything is synchronous and single-threaded, so only a single remote node connection at a time is allowed. -- discovery (DHT, multicast-DNS, etc) is not implemented. ### Dependencies diff --git a/TODO b/TODO index 4add30c..f1ccb08 100644 --- a/TODO +++ b/TODO @@ -1,19 +1,14 @@ next: +- clean up node bitfield implementation +- "insert" API for HyperRegister - crude single-thread drive+network clone +- sparse register clone +- "needed data chunks" and progress function? - crude single-thread drive+network broadcast -- drive: checkout all latest files -- SLEEP bitfields stuff -- .latest and .ogd files -- sparse insert into SLEEP/drive -- secret_key-in-home helpers (read/write) -- multi-connection network sync (per-connection state, etc) -- duplicate file/chunk optimizations - 'geniza clone' command - 'geniza checkout' command - 'geniza init', 'geniza commit' commands -- 'geniza broadcast' command -- API to link and run from, eg, python specific tests: - drive children array: deep directory nesting (like 'tree' example) @@ -25,20 +20,20 @@ specific tests: - cloning with metadata pointing to relative or absolute paths ("dir/../../../../etc/passwd") -trivial performance: +performance: - cache SLEEP file length when opened (don't fstat all the time) - keep an LRU cache of SLEEP reads -APIs: -- trait: register -- trait: drive -- SLEEP file -- SLEEP register -- SLEEP drive -- peer (multiple transit types) -- synchronizer (register, peers, discovery) - Backburner: +- API to link and run from, eg, python +- uTP transport +- in-memory storage backend +- WASM +- multi-connection network sync (per-connection state, etc) +- duplicate file/chunk optimizations +- secret_key-in-home helpers (read/write) +- SLEEP bitfields stuff +- .latest and .ogd files - benchmarks! syscall counts (strace), timing, etc - test fault injection... with strace? - consider kafka (append performance) and sqlite (robustness) file I/O patterns -- cgit v1.2.3