aboutsummaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
authorBryan Newbold <bnewbold@robocracy.org>2017-11-24 18:49:52 -0800
committerBryan Newbold <bnewbold@robocracy.org>2017-11-24 18:50:15 -0800
commit82a4ce577b9da17e7eccc78b04b7e2e96b809b75 (patch)
tree2c64d48f0caefa06136bbc499e9226d7776326c3
parent0dae9ee7057145fdc6761199f0292d103f5bd0c4 (diff)
downloadgeniza-82a4ce577b9da17e7eccc78b04b7e2e96b809b75.tar.gz
geniza-82a4ce577b9da17e7eccc78b04b7e2e96b809b75.zip
notes/updates
-rw-r--r--README.md2
-rw-r--r--TODO42
-rw-r--r--notes/bugs.txt6
-rw-r--r--notes/crypto.txt5
-rw-r--r--notes/drive.md7
-rw-r--r--notes/typos.txt4
6 files changed, 42 insertions, 24 deletions
diff --git a/README.md b/README.md
index 25a8169..f6a06fc 100644
--- a/README.md
+++ b/README.md
@@ -28,7 +28,7 @@ it will eat-your-data!
- [x] read file tree ("ls")
- [x] import file to register
- [x] export file from register
- - [ ] import/export directories recursively
+ - [x] import/export directories recursively
- [ ] Networking
- [x] send/receive encrypted messages to a known host
- [ ] receive entire register from a known host
diff --git a/TODO b/TODO
index 7663eb3..4add30c 100644
--- a/TODO
+++ b/TODO
@@ -1,25 +1,22 @@
next:
-- file check in/out
-- drive: deletion
-- drive: think through child index creation
-- drive+network sync
-- drive+network broadcast
-- bitfield stuff (un-indexed)
- https://docs.rs/bit_field/0.8.0/bit_field/trait.BitArray
+- crude single-thread drive+network clone
+- crude single-thread drive+network broadcast
+- drive: checkout all latest files
+- SLEEP bitfields stuff
+- .latest and .ogd files
+- sparse insert into SLEEP/drive
- secret_key-in-home helpers (read/write)
+- multi-connection network sync (per-connection state, etc)
+- duplicate file/chunk optimizations
- 'geniza clone' command
- 'geniza checkout' command
- 'geniza init', 'geniza commit' commands
- 'geniza broadcast' command
- API to link and run from, eg, python
-meta:
-- travis tests (linux, osx)
-
-misc:
-
specific tests:
+- drive children array: deep directory nesting (like 'tree' example)
- sleep info a variety of bad sleep files, should error
- sleep create a known file, should be same as empty (also a "Rot13" algo file)
- create a register, binary compare against known-good
@@ -28,6 +25,10 @@ specific tests:
- cloning with metadata pointing to relative or absolute paths
("dir/../../../../etc/passwd")
+trivial performance:
+- cache SLEEP file length when opened (don't fstat all the time)
+- keep an LRU cache of SLEEP reads
+
APIs:
- trait: register
- trait: drive
@@ -37,23 +38,13 @@ APIs:
- peer (multiple transit types)
- synchronizer (register, peers, discovery)
-Old API:
-sleep: SLEEP file I/O
-register: hypercore registers/journal streams, signing
-drive: hyperdrive
-sync: dat network protocol, discover
-geniza: command line, swarm
-
-
-
Backburner:
- benchmarks! syscall counts (strace), timing, etc
- test fault injection... with strace?
- consider kafka (append performance) and sqlite (robustness) file I/O patterns
- manpage
- slog for logging
-- lru caches
-- disable sodiumoxide serde feature
+- disable sodiumoxide serde feature (faster/smaller build)
- sodiumoxide static link
https://safenetforum.org/t/building-a-non-dynamically-linked-executable-of-safe-vault/9811/6
- SLEEP API: should just be empty string for algo-name, not Option<>
@@ -70,3 +61,8 @@ Backburner:
- turn magic numbers in tree nodes into declared (public?) constants
- root_nodes as an iterator? (vs returning a Vec)
- pretty terminal output: https://crates.io/crates/colored
+- error handling best practices:
+ https://rust-lang-nursery.github.io/rust-cookbook/basics.html#obtain-backtrace-of-complex-error-scenarios
+- debug commands with hex i/o:
+ https://github.com/ia0/data-encoding
+- sendfile() or splice() for disk-to-disk copying?
diff --git a/notes/bugs.txt b/notes/bugs.txt
new file mode 100644
index 0000000..c4fa6fa
--- /dev/null
+++ b/notes/bugs.txt
@@ -0,0 +1,6 @@
+
+possible upstream bug:
+ => sync data in subdir
+ => stop client
+ => remove entire dir
+ => doesn't `del` on later sync
diff --git a/notes/crypto.txt b/notes/crypto.txt
index ea81a5f..0a80619 100644
--- a/notes/crypto.txt
+++ b/notes/crypto.txt
@@ -7,3 +7,8 @@ https://nacl.cr.yp.to/stream.html
https://github.com/maidsafe/rust_sodium
https://groups.google.com/forum/#!original/boring-crypto/BpUmNMXKMYQ/EEwAIeQdjacJ
+
+TLS is a pain, but if it (or some other standard) was used instead of the
+current libsodium, it might be possible in the future to get kernel-mode
+encryption going.
+
diff --git a/notes/drive.md b/notes/drive.md
index 8148acf..9116117 100644
--- a/notes/drive.md
+++ b/notes/drive.md
@@ -18,3 +18,10 @@ The current path counts as an entry (eg, "/README" decomposes to "/" and
Entry indexes are effectively 1-based (0 is the pubkey entry); don't correct
for this.
+
+In children:
+- Changes/edits to a path entirely replace the old entry index with the new
+ index.
+- removals of an entry are done by writing a new entry with the same path, no
+ stat data, and with all child paths removed (of this index and the old
+ index).
diff --git a/notes/typos.txt b/notes/typos.txt
new file mode 100644
index 0000000..2198381
--- /dev/null
+++ b/notes/typos.txt
@@ -0,0 +1,4 @@
+
+"check if all bites in the 2 bytes"
+
+"compresses nicely as its all contiguous ones on disk"