1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
|
next:
- DatConnection
=> u32 for feed index
=> change sockaddr argument... not sure to what, maybe ToSockAddr?
=> rip out more crap ("receive some", no_delay, registration extras)
=> how to accept() (from_stream() refactor?)
- PeerThread
=> proper error propagation
=> don't store state (bitfield) in PeerThread itself, leave to Synchronizer
=> to start, just use feed index (don't keep key/index table)
=> need "self peer id" passed around
=> keep "peer handle"... in the future, should be based on remote handle returned
- Synchronizer
=> probably an options builder
=> generate "self peer id"
=> "wanted" bitmap
=> design statemachine (global and per-peer)
=> actions for each message type
=> PeerState helper, with bitfield
=> when a peer is ready, find a mutal chunk option, mark as in progress
=> some logic to do drive stuff (add new registers, etc)
=> chan-signal to shutdown
- get everything to compile
- clean up node bitfield implementation
=> use bit_vec (bit-vec = "0.4.4") instead of bit_field (remove)
- "insert" API for HyperRegister (when we have a signed Data message)
- how to create a Data message in response to a Request
- sparse register clone
- 'geniza clone' command
- 'geniza checkout' command
- 'geniza init', 'geniza commit' commands
threaded sync:
- rust stdlib async TCP?
- event loop runner
- channel polling
- polling for connections (channels + async TCP)
- UNIX signal handling
- later: accepting inbound connections
specific tests:
- drive children array: deep directory nesting (like 'tree' example)
- sleep info a variety of bad sleep files, should error
- sleep create a known file, should be same as empty (also a "Rot13" algo file)
- create a register, binary compare against known-good
- test vectors of leaf, parent, and root hashes
- test vectors of signatures
- cloning with metadata pointing to relative or absolute paths
("dir/../../../../etc/passwd")
performance:
- cache SLEEP file length when opened (don't fstat all the time)
- keep an LRU cache of SLEEP reads
Backburner:
- try switching to Coded{Input/Output}Stream for fewer copies/allocations
- API to link and run from, eg, python
- uTP transport
- in-memory storage backend
- compile to WASM... play in browser?
- multi-connection network sync (per-connection state, etc)
- duplicate file/chunk optimizations
- secret_key-in-home helpers (read/write)
- SLEEP bitfields stuff
- .latest and .ogd files
- benchmarks! syscall counts (strace), timing, etc
- test fault injection... with strace?
- consider kafka (append performance) and sqlite (robustness) file I/O patterns
- manpage
- slog for logging
- disable sodiumoxide serde feature (faster/smaller build)
- sodiumoxide static link
https://safenetforum.org/t/building-a-non-dynamically-linked-executable-of-safe-vault/9811/6
- SLEEP API: should just be empty string for algo-name, not Option<>
- I have the SLEEP/register naming wrong... sleep refers to the directory?
- portable positional file I/O
https://github.com/vasi/positioned-io
- mmap
https://docs.rs/scroll/0.7.0/scroll/
- switch to byteorder for endian reads/writes
- --json args to most CLI commands
- ruthlessly simplify APIs
- SleepFile from existing File (api)
- refactor key generation into generic/re-usable function
- turn magic numbers in tree nodes into declared (public?) constants
- root_nodes as an iterator? (vs returning a Vec)
- pretty terminal output: https://crates.io/crates/colored
- error handling best practices:
https://rust-lang-nursery.github.io/rust-cookbook/basics.html#obtain-backtrace-of-complex-error-scenarios
- debug commands with hex i/o:
https://github.com/ia0/data-encoding
- sendfile() or splice() for disk-to-disk copying?
- Rust FS API example: https://docs.rs/zbox/0.1.3/zbox/
- "bytes" crate
|
