diff options
| -rw-r--r-- | README.md | 2 | ||||
| -rw-r--r-- | TODO | 42 | ||||
| -rw-r--r-- | notes/bugs.txt | 6 | ||||
| -rw-r--r-- | notes/crypto.txt | 5 | ||||
| -rw-r--r-- | notes/drive.md | 7 | ||||
| -rw-r--r-- | notes/typos.txt | 4 |
6 files changed, 42 insertions, 24 deletions
@@ -28,7 +28,7 @@ it will eat-your-data! - [x] read file tree ("ls") - [x] import file to register - [x] export file from register - - [ ] import/export directories recursively + - [x] import/export directories recursively - [ ] Networking - [x] send/receive encrypted messages to a known host - [ ] receive entire register from a known host @@ -1,25 +1,22 @@ next: -- file check in/out -- drive: deletion -- drive: think through child index creation -- drive+network sync -- drive+network broadcast -- bitfield stuff (un-indexed) - https://docs.rs/bit_field/0.8.0/bit_field/trait.BitArray +- crude single-thread drive+network clone +- crude single-thread drive+network broadcast +- drive: checkout all latest files +- SLEEP bitfields stuff +- .latest and .ogd files +- sparse insert into SLEEP/drive - secret_key-in-home helpers (read/write) +- multi-connection network sync (per-connection state, etc) +- duplicate file/chunk optimizations - 'geniza clone' command - 'geniza checkout' command - 'geniza init', 'geniza commit' commands - 'geniza broadcast' command - API to link and run from, eg, python -meta: -- travis tests (linux, osx) - -misc: - specific tests: +- drive children array: deep directory nesting (like 'tree' example) - sleep info a variety of bad sleep files, should error - sleep create a known file, should be same as empty (also a "Rot13" algo file) - create a register, binary compare against known-good @@ -28,6 +25,10 @@ specific tests: - cloning with metadata pointing to relative or absolute paths ("dir/../../../../etc/passwd") +trivial performance: +- cache SLEEP file length when opened (don't fstat all the time) +- keep an LRU cache of SLEEP reads + APIs: - trait: register - trait: drive @@ -37,23 +38,13 @@ APIs: - peer (multiple transit types) - synchronizer (register, peers, discovery) -Old API: -sleep: SLEEP file I/O -register: hypercore registers/journal streams, signing -drive: hyperdrive -sync: dat network protocol, discover -geniza: command line, swarm - - - Backburner: - benchmarks! syscall counts (strace), timing, etc - test fault injection... with strace? - consider kafka (append performance) and sqlite (robustness) file I/O patterns - manpage - slog for logging -- lru caches -- disable sodiumoxide serde feature +- disable sodiumoxide serde feature (faster/smaller build) - sodiumoxide static link https://safenetforum.org/t/building-a-non-dynamically-linked-executable-of-safe-vault/9811/6 - SLEEP API: should just be empty string for algo-name, not Option<> @@ -70,3 +61,8 @@ Backburner: - turn magic numbers in tree nodes into declared (public?) constants - root_nodes as an iterator? (vs returning a Vec) - pretty terminal output: https://crates.io/crates/colored +- error handling best practices: + https://rust-lang-nursery.github.io/rust-cookbook/basics.html#obtain-backtrace-of-complex-error-scenarios +- debug commands with hex i/o: + https://github.com/ia0/data-encoding +- sendfile() or splice() for disk-to-disk copying? diff --git a/notes/bugs.txt b/notes/bugs.txt new file mode 100644 index 0000000..c4fa6fa --- /dev/null +++ b/notes/bugs.txt @@ -0,0 +1,6 @@ + +possible upstream bug: + => sync data in subdir + => stop client + => remove entire dir + => doesn't `del` on later sync diff --git a/notes/crypto.txt b/notes/crypto.txt index ea81a5f..0a80619 100644 --- a/notes/crypto.txt +++ b/notes/crypto.txt @@ -7,3 +7,8 @@ https://nacl.cr.yp.to/stream.html https://github.com/maidsafe/rust_sodium https://groups.google.com/forum/#!original/boring-crypto/BpUmNMXKMYQ/EEwAIeQdjacJ + +TLS is a pain, but if it (or some other standard) was used instead of the +current libsodium, it might be possible in the future to get kernel-mode +encryption going. + diff --git a/notes/drive.md b/notes/drive.md index 8148acf..9116117 100644 --- a/notes/drive.md +++ b/notes/drive.md @@ -18,3 +18,10 @@ The current path counts as an entry (eg, "/README" decomposes to "/" and Entry indexes are effectively 1-based (0 is the pubkey entry); don't correct for this. + +In children: +- Changes/edits to a path entirely replace the old entry index with the new + index. +- removals of an entry are done by writing a new entry with the same path, no + stat data, and with all child paths removed (of this index and the old + index). diff --git a/notes/typos.txt b/notes/typos.txt new file mode 100644 index 0000000..2198381 --- /dev/null +++ b/notes/typos.txt @@ -0,0 +1,4 @@ + +"check if all bites in the 2 bytes" + +"compresses nicely as its all contiguous ones on disk" |