aboutsummaryrefslogtreecommitdiffstats
path: root/TODO
blob: 9a008ad07c7d571a1057400592fac38869b3c924 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95

next:
- DatConnection
    => u32 for feed index
    => change sockaddr argument... not sure to what, maybe ToSockAddr?
    => rip out more crap ("receive some", no_delay, registration extras)
    => how to accept() (from_stream() refactor?)
- PeerThread
    => proper error propagation
    => don't store state (bitfield) in PeerThread itself, leave to Synchronizer
    => to start, just use feed index (don't keep key/index table)
    => need "self peer id" passed around
    => keep "peer handle"... in the future, should be based on remote handle returned
- Synchronizer
    => probably an options builder
    => generate "self peer id"
    => "wanted" bitmap
    => design statemachine (global and per-peer)
    => actions for each message type
    => PeerState helper, with bitfield
    => when a peer is ready, find a mutal chunk option, mark as in progress
    => some logic to do drive stuff (add new registers, etc)
    => chan-signal to shutdown
- get everything to compile
- clean up node bitfield implementation
    => use bit_vec (bit-vec = "0.4.4") instead of bit_field (remove)
- "insert" API for HyperRegister (when we have a signed Data message)
- how to create a Data message in response to a Request
- sparse register clone
- 'geniza clone' command
- 'geniza checkout' command
- 'geniza init', 'geniza commit' commands

threaded sync:
- rust stdlib async TCP?
- event loop runner
- channel polling
- polling for connections (channels + async TCP)
- UNIX signal handling
- later: accepting inbound connections

specific tests:
- drive children array: deep directory nesting (like 'tree' example)
- sleep info a variety of bad sleep files, should error
- sleep create a known file, should be same as empty (also a "Rot13" algo file)
- create a register, binary compare against known-good
- test vectors of leaf, parent, and root hashes
- test vectors of signatures
- cloning with metadata pointing to relative or absolute paths
  ("dir/../../../../etc/passwd")

performance:
- cache SLEEP file length when opened (don't fstat all the time)
- keep an LRU cache of SLEEP reads

Backburner:
- try switching to Coded{Input/Output}Stream for fewer copies/allocations
- API to link and run from, eg, python
- uTP transport
- in-memory storage backend
- compile to WASM... play in browser?
- multi-connection network sync (per-connection state, etc)
- duplicate file/chunk optimizations
- secret_key-in-home helpers (read/write)
- SLEEP bitfields stuff
- .latest and .ogd files
- benchmarks! syscall counts (strace), timing, etc
- test fault injection... with strace?
- consider kafka (append performance) and sqlite (robustness) file I/O patterns
- manpage
- slog for logging
- disable sodiumoxide serde feature (faster/smaller build)
- sodiumoxide static link
    https://safenetforum.org/t/building-a-non-dynamically-linked-executable-of-safe-vault/9811/6
- SLEEP API: should just be empty string for algo-name, not Option<>
- I have the SLEEP/register naming wrong... sleep refers to the directory?
- portable positional file I/O
    https://github.com/vasi/positioned-io
- mmap
    https://docs.rs/scroll/0.7.0/scroll/
- switch to byteorder for endian reads/writes
- --json args to most CLI commands
- ruthlessly simplify APIs
- SleepFile from existing File (api)
- refactor key generation into generic/re-usable function
- turn magic numbers in tree nodes into declared (public?) constants
- root_nodes as an iterator? (vs returning a Vec)
- pretty terminal output: https://crates.io/crates/colored
- error handling best practices:
    https://rust-lang-nursery.github.io/rust-cookbook/basics.html#obtain-backtrace-of-complex-error-scenarios
- debug commands with hex i/o:
    https://github.com/ia0/data-encoding
- sendfile() or splice() for disk-to-disk copying?
- Rust FS API example: https://docs.rs/zbox/0.1.3/zbox/
- "bytes" crate