libsodium Ed25519 keys, Blake2b 32-bit hashes This file is a WIP. The 'simple' example might have errors; the hypercore tests and 'alphabet' example seemed to actually work (aka, matches dat upstream client behavior). ====== simple bnewbold@orithena$ cat test-data/dat/simple/.dat/content.key | base64 BnlFLuVBAS6rYID8wUCmIS/O85v/jJ9i7ousH16C0r0= metadata pub: len_bytes: 32 hex: cb1e8d65f3c6242db90bfd99accaf47930ea3ff6c852fb50caaa548c9cdcb8b7 base64: yx6NZfPGJC25C/2ZrMr0eTDqP/bIUvtQyqpUjJzcuLc= metadata secret: len_bytes: 64 file: ~/.dat/secret_keys/bf/feaefe7cb34458e11ecb1e38b67b5f93eafce2bd7a286db229a87f0384e61c hex: ab3bd2909310796c36ac66f85bec5b18d4e09397449225623adab99e23a41224cb1e8d65f3c6242db90bfd99accaf47930ea3ff6c852fb50caaa548c9cdcb8b7 base64: qzvSkJMQeWw2rGb4W+xbGNTgk5dEkiViOtq5niOkEiTLHo1l88YkLbkL/ZmsyvR5MOo/9shS+1DKqlSMnNy4tw== discovery key: len_bytes: 32 hex: bffeaefe7cb34458e11ecb1e38b67b5f93eafce2bd7a286db229a87f0384e61c content pub: len_bytes: 32 hex: 0679452ee541012eab6080fcc140a6212fcef39bff8c9f62ee8bac1f5e82d2bd base64: BnlFLuVBAS6rYID8wUCmIS/O85v/jJ9i7ousH16C0r0= content secret: len_bytes: 64 hex: c322b4918b90e53f13ab14c7e86dec7e5e7bd4128b585dac6bdf0353f8e4974f0679452ee541012eab6080fcc140a6212fcef39bff8c9f62ee8bac1f5e82d2bd base64: wyK0kYuQ5T8TqxTH6G3sfl571BKLWF2sa98DU/jkl08GeUUu5UEBLqtggPzBQKYhL87zm/+Mn2Lui6wfXoLSvQ== To make a discovery key, do a "keyed" Blake2b hash of the string "hypercore" (9 bytes, no null) using the public key. bnewbold@orithena$ ./target/debug/geniza-sleep file-read-all test-data/dat/simple/.dat/content.tree 0: debug: Ok([103, 23, 156, 36, 59, 56, 123, 124, 124, 66, 11, 215, 189, 198, 157, 53, 192, 97, 185, 121, 187, 54, 95, 156, 192, 123, 69, 39, 238, 221, 238, 250, 0, 0, 0, 0, 0, 0, 0, 66]) hex: 67179c243b387b7c7c420bd7bdc69d35c061b979bb365f9cc07b4527eeddeefa0000000000000042 1: debug: Ok([43, 235, 230, 134, 54, 24, 197, 134, 8, 109, 37, 32, 170, 13, 19, 218, 102, 107, 0, 46, 90, 210, 177, 98, 35, 161, 91, 193, 85, 134, 241, 228, 0, 0, 0, 0, 0, 0, 0, 204]) hex: 2bebe6863618c586086d2520aa0d13da666b002e5ad2b16223a15bc15586f1e400000000000000cc 2: debug: Ok([163, 78, 52, 182, 129, 66, 214, 34, 130, 211, 196, 230, 152, 173, 250, 133, 82, 164, 113, 127, 75, 106, 60, 76, 179, 136, 210, 137, 113, 45, 233, 250, 0, 0, 0, 0, 0, 0, 0, 138]) hex: a34e34b68142d62282d3c4e698adfa8552a4717f4b6a3c4cb388d289712de9fa000000000000008a Signature file should be 64 byte each: bnewbold@orithena$ ./target/debug/geniza-sleep file-read-all test-data/dat/simple/.dat/content.signatures 0: debug: Ok([235, 70, 69, 206, 179, 128, 50, 76, 255, 139, 94, 124, 133, 55, 170, 130, 46, 139, 101, 250, 234, 52, 122, 184, 195, 40, 149, 95, 215, 220, 160, 39, 99, 173, 22, 146, 226, 165, 213, 245, 57, 186, 81, 45, 159, 218, 248, 109, 140, 172, 99, 57, 155, 180, 128, 17, 181, 220, 90, 22, 238, 78, 81, 1]) hex: eb4645ceb380324cff8b5e7c8537aa822e8b65faea347ab8c328955fd7dca02763ad1692e2a5d5f539ba512d9fdaf86d8cac63399bb48011b5dc5a16ee4e5101 1: debug: Ok([53, 69, 157, 171, 205, 82, 230, 197, 158, 124, 63, 2, 46, 165, 192, 24, 86, 33, 58, 126, 220, 80, 116, 202, 6, 194, 40, 108, 160, 63, 63, 245, 248, 171, 2, 76, 207, 153, 236, 176, 192, 193, 17, 45, 64, 246, 160, 228, 9, 120, 117, 115, 209, 34, 91, 117, 210, 82, 15, 238, 21, 160, 151, 9]) hex: 35459dabcd52e6c59e7c3f022ea5c01856213a7edc5074ca06c2286ca03f3ff5f8ab024ccf99ecb0c0c1112d40f6a0e409787573d1225b75d2520fee15a09709 Paper says: Ed25519 sign( BLAKE2b( <1 byte> 2 // root type for (every root node left-to-right) { <32 byte root hash> <8 byte Uint64BE root tree index> <8 byte Uint64BE child byte lengths> } ) ) =========== example from hypercore tests Data chunks are just 'a', 'b', 'c'. Public key: 9718a1ff1c4ca79feac551c0c7212a65e4091278ec886b88be01ee4039682238 Secret key: 53729c0311846cca9cc0eded07aaf9e6689705b6a0b1bb8c3a2a839b72fda3839718a1ff1c4ca79feac551c0c7212a65e4091278ec886b88be01ee4039682238 Tree: 0: ab27d45f509274ce0d08f4f09ba2d0e0d8df61a0c2a78932e81b5ef26ef398df0000000000000001 1: 064321a8413be8c604599689e2c7a59367b031b598bceeeb16556a8f3252e0de0000000000000002 2: 94c17054005942a002c7c39fbb9c6183518691fb401436f1a2f329b380230af80000000000000001 3: 00000000000000000000000000000000000000000000000000000000000000000000000000000000 4: 94c17054005942a002c7c39fbb9c6183518691fb401436f1a2f329b380230af80000000000000001 BLAKE2b ( 00, 00 00 00 00 00 00 00 01, 61, ) = ab27d45f509274ce0d08f4f09ba2d0e0d8df61a0c2a78932e81b5ef26ef398df Signatures: 0: 84684e8dd76c339d6f5754e813204906ee818e6c6cdc6a816a2ac785a3e0d926ac08641a904013194fe6121847b7dad4e361965d47715428eb0a0ededbdd5909 1: d037ff3c3614fa0100ed9264a712d3b77cbe7a4f6eadd8f342809be99dfb9154a19e278d7a5de7d2b4d890f7701a38b006469f6bab1aff66ac6125d48bafdc07 2: 11057675ed57d445ce7ed4613881be37ebc56bb40556b822e431bb4dc3517421f9a5e3ed124eb5c4db8367386d9ce12b2408613b9fec2837022772a635ffd807 Ed25519 sign( BLAKE2b ( 02, ab27d45f509274ce0d08f4f09ba2d0e0d8df61a0c2a78932e81b5ef26ef398df, 00 00 00 00 00 00 00 00, 00 00 00 00 00 00 00 01, ) = fd09e68350db613d3afc9390abf12a7c2693d602b69012ff068251568d05887b ) = 84684e8dd76c339d6f5754e813204906ee818e6c6cdc6a816a2ac785a3e0d926ac08641a904013194fe6121847b7dad4e361965d47715428eb0a0ededbdd5909 Cool! It works! ====== alphabet Tree: 0: ab27d45f509274ce0d08f4f09ba2d0e0d8df61a0c2a78932e81b5ef26ef398df0000000000000001 1: 064321a8413be8c604599689e2c7a59367b031b598bceeeb16556a8f3252e0de0000000000000002 2: 94c17054005942a002c7c39fbb9c6183518691fb401436f1a2f329b380230af80000000000000001 ab27d45f509274ce0d08f4f09ba2d0e0d8df61a0c2a78932e81b5ef26ef398df0000000000000001 588e663097be0b31a3862613f96a1826be03c4ee529f11a495d193465d42b4ab0000000000000002 <= AH HA! parent hashing 064321a8413be8c604599689e2c7a59367b031b598bceeeb16556a8f3252e0de0000000000000002 Ah, the gotcha: when making parent hashes, make sure to only include child *hashes* (32-bytes), not entire child entries (40 bytes, including lengths). Ah, and signatures were failing because I was passing an index to tree_root_nodes(), not an entry_count (index+1).