path: root/adenosine-pds/plan.txt

PDS proof of concept:
x ipld sqlite driver importing CAR file
    => simple binary, two args
x skeleton
  x env config: DB paths, port
  x commands: serve, import, inspect
  x integration test
  x atp db wrapper (with methods)
        schema in a single .sql file
        https://docs.rs/rusqlite_migration/latest/rusqlite_migration/
        test version (in memory, per-thread)
        wrap in a mutex, unwrap and make new connection when desired
  x wrap both database in a struct with mutexes; have "get handle" helper that unlocks and returns a connection copy of the given type
  x repo store database wrapper (with methods)
  x response error handling (especially for XRPC endpoints)
x basic crypto and did:plc stuff
  x did:key read/write helpers
  x test that did:plc generated as expected
  x signature read/write helpers
x single shared signing key for all users (not what I expected)
x sqlite schema (for application)
x fix did multibase key encoding:
    https://medium.com/asecuritysite-when-bob-met-alice/02-03-or-04-so-what-are-compressed-and-uncompressed-public-keys-6abcb57efeb6
x MST code to read and mutate tree state
    x  check that empty tree works (eg, for account creation, and after deletes)
    x  with in-memory tests
    x  mutation batches
x IPLD objects to JSON value
x Did, Tid, Nsid types, with TID generator object?
x push types back "down" (less String typing in function signatures)
    => eg Cid in repo
x helper web methods
x cli: aturi should accept trailing slash
x implement basic non-authenticated CRUD on repository, test with CLI
    com.atproto
      x getAccountsConfig
      x createAccount
      x repoGetRecord
      x repoListRecords
      x syncGetRoot
      x repoDescribe
      x createSession
      x repoBatchWrite
      x     repoCreateRecord
      x     repoPutRecord
      x     repoDeleteRecord
      x syncGetRepo
      x syncUpdateRepo
x service-level config
      x domain suffixes (eg, just ".test" for now)
      x account registration allowed or not
      x PDS signing key
x CLI account creation (including did:web accounts)
- HTTP API handler implementing many endpoints
    com.atproto
      x getSession
      x resolveName
    app.bsky
      x updateProfile
      x getProfile

      x getHomeFeed
      x getAuthorFeed
        getPostThread

        getUserFollowers
        getUserFollows

        getLikedBy
        getRepostedBy

        getNotifications
        getNotificationCount
        postNotificationsSeen

        getUsersSearch

x cli: 'bsky timeline' command (for self)
x wire up basic bsky stuff to web interface
x bsky app handler (new module)
    x  batch mutation handler
    x  high-level database/repo helpers
x post threads (simple parent/children to start)

- basic CLI testing of bsky stuff (against upstream version and mine)
    => follow
    => like
    => follow
    => repost
    => author feed (of their posts and reposts)
    => timeline feed
- bit of polish
    x  web error handler (request to request)
    x  handlers should catch mutex poison error and exist process
    x  /about
    x  version in web footer
    x  actually implement /.well-known/did.json, etc
    x  check if CAR export works (size, blocks)
    x  invite code validation
    x  registration domain validation (including lack of domain blocking all registration)
    x  update README
    x  basic manpage for adenosine-pds (just a stub)
    x  CHANGELOG
- push demo
    => systemd unit files
    => sketch nginx file
    => get single-domain stuff working on adze
    => get multi-domain stuff working on adze
    => post some example dummy content

    => git/local 0.1.0 tag
    => build and push debs to adze, some archive.org mirror location

    => share links

- v0.1.1 (or v0.2.0?)
    - make profile DID link an aturi (pink)
    - reverse order of author feed (?)
    - check compatibility with current atproto
    - CLI: update TID implementation
    - static compilation: rustls, musl
    - reduce clap and reqwest size?
    - PDS: RSS feed (and link to it)
    - PDS: export download link (on repository page?)
    - CLI: update profile
    - CLI: status actually connects to server, verifies auth

x basic web handler (separate plan)
- JSON schema type generation (separate crate?)

- RSS, webfinger, maybe some other protocols
    https://lib.rs/crates/rss

- improve updateRepo implementation
- figure out auth JWT 'sub' situation (not a UCAN? CLI should ignore?)
- switch to Argon2 for passwords? meh
- aturi canonicalization helper (re-writes usernames to DID?)
- python XRPC API test script
- PDS CLI helpers
    create-account <username> <password> <email>
    reset-password <username> <password>
    list-repos
    list-accounts 
    import-car <car-file> [<did|alias>}

later:
x TODO: why are the multiformat keys so long in did doc?
- correct JWT helper stuff (?)
- did:web handler?

more PDS CLI helpers
    reindex <did>
        => locally re-generate cache tables
    update-repo <did> [<pds-server>] [<plc-server>]
        => or some full URI or something?
    update <did> [<plc-server>]
        => does this work yet?
    update-all-dids
        => would re-fetch did cods for all non-local users (?)
    spider
        => collect a big list of DIDs
        => fetch all the remote repos for those DIDs

other utils/helpers:
- pack/unpack a repo CAR into JSON files in a directory tree (plus a commit.json with sig?)

libraries:
- `jsonschema` to validate requests and records (rich validation)
- `schemafy` to codegen serde types for records (ahead of time?)
- pretty_env_logger
- no good published crate for working with CAR files... could rip out this code?
    https://github.com/n0-computer/iroh/tree/main/iroh-car
- ??? for CBOR (de)serialization of MST, separate from the IPLD stuff?

sync option:
- `rouille` web framework
- `rusqlite` with "bundled" sqlite for datastore
- `rusqlite_migration`
- `ipfs-sqlite-block-store` and `libipld` to parse and persist repo content 

async option:
- `warp` as async HTTP service
- `sqlx` for async pooled sqlite or postgresql db
- `iroh-store` for async rocksdb IPFS blockstore

## concurrency (in warp app)

note that there isn't really any point in having this be async, given that we
just have a single shared sqlite on disk. could try `rouille` instead of
`warp`?

maybe good for remote stuff like did:web resolution?

could try using sqlx instead of rusqlite for natively-async database stuff?

for block store:
- open a single connection at startup, store in mutex
- handlers get a reference to mutex. if they need a connection, they enter a blocking thread then:
    block on the mutex, then create a new connection, unlock the mutex
    do any operations on connection synchronously
    exit the block

## system tables

account
    did (PK)
    username (UNIQUE, indexed)
    email (UNIQUE)
    password_bcrypt
    signing_key

did_doc
    did (PK)
    seen_at (timestamp)

session
    did
    jwt
    ???

repo
    did
    head_commit

record (should this exist? good for queries)
    did
    collection
    tid
    record_cid
    record_cbor (CBOR bytes? JSON?)

password_reset
    did
    token


## atp tables

what actually needs to be indexed?
- post replies (forwards and backwards
- likes (back index)
- follows (back index)
- usernames (as part of profile?)
- mentions? hashtags?

additional state
- notifications

bsky_post
    did
    tid (or timestamp from tid?)
    text
    reply_root (nullable)
    reply_parent (nullable)
    entities: JSON (?)

bsky_profile
    did
    tid
    display_name
    description
    my_state (JSON)

bsky_follow
    did
    tid
    target_did

bsky_like
    did
    tid
    target_uri
    target_cid (what is this? the commit, or record CID?)

bsky_repost
    did
    tid
    target_uri
    target_cid

bsky_notification
    did
    created_at (timestamp)
    seen (boolean)
    reason
    target_uri

TODO:
- bsky_badge (etc)
- bsky_media_embed


----

what is needed to be useful?
- workflow to self-hosting a single user with did:web
- host multiple users with did:plc and wildcard domain
- allow users to bring their own did:web identifier, which is not controlled directly by the PDS
    => some method to register with existing DID
    => export did doc
- some mechanism of automatically pulling from other PDS

what might be interesting?
- basic read-only web interface
    => tech "explorer" for MST etc
- RSS feeds
- activitypub export
- PDS CLI that can inspect tree structure