diff options
| -rw-r--r-- | CHANGELOG.md | 22 | ||||
| -rw-r--r-- | README.md | 89 | ||||
| -rw-r--r-- | adenosine-cli/README.md | 116 | ||||
| -rw-r--r-- | adenosine-pds/README.md | 101 |
4 files changed, 209 insertions, 119 deletions
diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 0000000..e4528d4 --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,22 @@ + +# CHANGELOG + +## [0.1.0] - 2022-11-11 + +First tagged release. + +Both the AT protocol and this project are very much a work in progress, and +there should be zero expectation of stability, backwards/forwards comatability, +or supported upgrade paths at this time. + +Initial features: + +- cli: blocking implementation with `reqwest` +- cli: generic XRPC methods (com.atproto Lexicon) +- cli: partial app.bsky Lexicon support +- pds: blocking implementation with `rouille`, `rusqlite`, `ipfs-sqlite-block-store` +- pds: crude repository storage (MST in IPLD blocks) +- pds: crude system and application database (sqlite) +- pds: basic read-only web interface to repository content, bsky profiles and posts +- pds: self-hosted did:web configuration +- pds: small-world did:plc with registration configuration @@ -8,7 +8,7 @@ `adenosine`: enthusiast-grade implementation of atproto.com in Rust =================================================================== -**Status:** it doesn't even work yet and will eat your data +**Status:** it doesn't really work yet and will eat your data This is a hobby project to implement components of the proposed Bluesky AT Protocol ([atproto.com](https://atproto.com)) for federated social media, as @@ -16,95 +16,22 @@ initially announced in Fall 2022. This might be interesting for other folks to take a spin with, but isn't intended to host real content from real people. The goal is to think through how the protocol might work by implementing it. -Components planned: +Components: -- `adenosine-cli`: command-line client (`adenosine`), partially implemented and working -- `adenosine-pds`: "small world" personal data server implementation, with data in sqlite, not implemented +- `adenosine-cli` ([README](./adenosine-cli/README.md): command-line client (`adenosine`) +- `adenosine-pds` ([README](./adenosine-pds/README.md): "small world" personal data server implementation, with data in sqlite -Not currently planning on implementing the `did:plc` method. Would just stick -with `did:web` for now, either manually placing `/.well-known/did.json` -documents on existing servers or trying a wildcard DNS hack. -## Quickstart (CLI) -To work with an actual PDS server, you will need to get the -[`bluesky-social/atproto`](https://github.com/bluesky-social/atproto) prototype -PDS up and running locally. There is a separate -[quickstart](./notes/atproto_quickstart.md) on how to get that going. +## Development and Contributions -If you run a Debian-base Linux distribution, you might be able to install an -existing package (TODO: link). Otherwise, you'll need to build the CLI tool -from source. Install a recent version of the Rust programming language -toolchain (eg, using [`rustup`](https://rustup.rs/) - -The top level workplace includes the GUI and PDS code, which include huge -dependency trees and are very slow to build. If you just want the CLI, enter -the `adenosine-cli` directory before running any `cargo` commands. - -To build, test, and locally install just the CLI (using `cargo`): - - cd adenosine-cli - cargo install - -To start interacting with a PDS, set the `ATP_HOST` environment variable. Then -either register a test account, or create a new session for an existing -account, and save the JWT token to the `ATP_AUTH_TOKEN`: - - # default port for bluesky-social/atproto implementation - export ATP_HOST=http://localhost:2583 - - # register a new account - adenosine account register -u voltaire.test -p bogus -e voltaire@example.com - { - "did": "did:plc:yqtuksvatmmgngd5nkkw75hn", - "jwt": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJkaWQ6cGxjOnlxdHVrc3ZhdG1tZ25nZDVua2t3NzVobiIsImlhdCI6MTY2Njk5NjMwNn0.MMQa4JIQdwvhy-rjJ0kO-z8-KdoOL0Lto9JtOkK-lwE", - "username": "voltaire.test" - } - - export ATP_AUTH_TOKEN=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJkaWQ6cGxjOnlxdHVrc3ZhdG1tZ25nZDVua2t3NzVobiIsImlhdCI6MTY2Njk5NjMwNn0.MMQa4JIQdwvhy-rjJ0kO-z8-KdoOL0Lto9JtOkK-lwE - - # to clear the auth token env variable - unset ATP_AUTH_TOKEN - - # create a new session (login) for existing account - adenosine account login -u voltaire.test -p bogus - { - "did": "did:plc:yqtuksvatmmgngd5nkkw75hn", - "jwt": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJkaWQ6cGxjOnlxdHVrc3ZhdG1tZ25nZDVua2t3NzVobiIsImlhdCI6MTY2Njk5NjQxNX0.j2wcF1g9NxT_1AvYRiplNf_jtK6S81y3L38AkcBwOqY", - "name": "voltaire.test" - } - - export ATP_AUTH_TOKEN=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJkaWQ6cGxjOnlxdHVrc3ZhdG1tZ25nZDVua2t3NzVobiIsImlhdCI6MTY2Njk5NjQxNX0.j2wcF1g9NxT_1AvYRiplNf_jtK6S81y3L38AkcBwOqY - -You could save the `ATP_HOST` and `ATP_AUTH_TOKEN` values in `~/.bashrc` so you -don't need to enter them every time. - -Now you can start posting and poking around: - - adenosine bsky post "gruel again for breakfast" - { - "cid": "bafyreig2aqlsg4arslck64wbo2hnhe6k2a4z3z2sjfzh3uapv3a4zjld7e", - "uri": "at://did:plc:yqtuksvatmmgngd5nkkw75hn/app.bsky.post/3jg5zkr322c2a" - } - - - adenosine ls at://voltaire.test - app.bsky.post - -There is a [manpage](./extra/adenosine.1.md) and [CLI-specific -README](./adenosine-cli/README.md) with more details and examples. - - -## Development - -Mininum Supported Rust Version (MSRV) is currently 1.61, using 2021 Rust +Minimum Supported Rust Version (MSRV) is currently 1.61, using 2021 Rust Edition. Contributions, conversations, bug reports, and handwriten postcards are all -welcome. As mentioned above this is a low-effort hobby project so high-touch -support, feature requests, growth strategies, etc, probably will not be -fielded. +welcome. This is a low-effort hobby project so high-touch support, feature +requests, growth strategies, etc, probably will not be fielded. The source code license is AGPLv3. Please acknowledge this in any initial contributions, and also indicate whether and how you would like to be diff --git a/adenosine-cli/README.md b/adenosine-cli/README.md index 6b526e8..c66479d 100644 --- a/adenosine-cli/README.md +++ b/adenosine-cli/README.md @@ -17,62 +17,110 @@ This client does not currently do any schema validation of Lexicons, either at compile time or runtime (eg, dynamically fetching Lexicons). The Bluesky Lexicon (bsky.app) is partially supported with helper commands. +There is a [manpage](../extra/adenosine.1.md) with usage details and examples. -## Example Commands +Features: -Generic atproto/XRPC commands: +- [x] generic XRPC invocation (GET and POST) +- [x] com.atproto Lexicon implementation +- [x] repo CAR file import/export +- [x] partial app.bsky Lexicon implementation +- [ ] complete app.bsky Lexicon implementation +- [ ] did:plc server interaction and chain validation +- [ ] pretty printing of bsky records (eg, timeline) with JSON as a flag +- [ ] test coverage - # configure host we are talking to - export ATP_HOST=http://localhost:2583 +Possible future features: - # register a new account - adenosine account register -u voltaire.test -p bogus -e voltaire@example.com - export ATP_AUTH_TOKEN=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJkaWQ6cGxjOm1qMzVmcmo3Ymd1NmMyd3N6YnRha3QzZSIsImlhdCI6MTY2NjkzNzQxMn0.908LeimAXg1txMMH4k0_IcZAVJaFw1k7pVkScGMNcmE +- [ ] Lexicon schema validation of records/requests +- [ ] DID resolution to repo location (independent of PDS) +- [ ] save login/configuration in homedir dotfile - # or, login - unset ATP_AUTH_TOKEN - adenosine account login -u voltaire.test -p bogus - export ATP_AUTH_TOKEN=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJkaWQ6cGxjOm1qMzVmcmo3Ymd1NmMyd3N6YnRha3QzZSIsImlhdCI6MTY2NjkzNzcyMn0.7lXJ9xl6c-hrzUAR9YGLc4iFBn4nOJPbFX8TmYDHgdE +## Installation - # check config and setup - adenosine status +`adenosine-cli` is implemented in Rust, and in theory could be distributed in +binary form for multiple platforms. + +TODO: debian packages for linux + +Otherwise, you'll need to build the CLI tool from source. Install a recent +version of the Rust programming language toolchain (eg, using +[`rustup`](https://rustup.rs/). + +The top level workplace includes the PDS code, which has a huge dependency tree +and is slow to build. If you just want the CLI, enter the `adenosine-cli` +directory before running any `cargo` commands: + + cd adenosine-cli + cargo install - # not implemented server-side (yet?) - adenosine account delete - adenosine account logout - # create, list, delete, update some records - adenosine create com.example.thing a=123 b=cream +## Quickstart - adenosine ls at://hyde.test/app.bsky.post - # TODO: bug in serve implementation? says "Could not find user" +You'll need an actual PDS server to interact with. As of November 2022, this +could be an independent implementation like `adenosine-pds`, or the more +official official [`bluesky-social/atproto`](https://github.com/bluesky-social/atproto) +prototype (see [local atproto dev quickstart](./../notes/atproto_quickstart.md)). - adenosine delete at://did:plc:mj35frj7bgu6c2wszbtakt3e/app.bsky.post/3jg4dqseut22a - adenosine describe +Set the `ATP_HOST` environment variable. Then either register a test account, +or create a new session for an existing account, and save the JWT token to the +`ATP_AUTH_TOKEN`: - adenosine get at://hyde.test/app.bsky.post/asdf + # default port for bluesky-social/atproto implementation + export ATP_HOST=http://localhost:2583 - adenosine resolve voltaire.test + # default port for adenosine-pds implementation + export ATP_HOST=http://localhost:3030 - adenosine repo root + # register a new account + adenosine account register -u voltaire.test -p bogus -e voltaire@example.com + { + "did": "did:plc:yqtuksvatmmgngd5nkkw75hn", + "jwt": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJkaWQ6cGxjOnlxdHVrc3ZhdG1tZ25nZDVua2t3NzVobiIsImlhdCI6MTY2Njk5NjMwNn0.MMQa4JIQdwvhy-rjJ0kO-z8-KdoOL0Lto9JtOkK-lwE", + "username": "voltaire.test" + } - adenosine repo root did:plc:mj35frj7bgu6c2wszbtakt3e + export ATP_AUTH_TOKEN=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJkaWQ6cGxjOnlxdHVrc3ZhdG1tZ25nZDVua2t3NzVobiIsImlhdCI6MTY2Njk5NjMwNn0.MMQa4JIQdwvhy-rjJ0kO-z8-KdoOL0Lto9JtOkK-lwE - adenosine repo export did:plc:mj35frj7bgu6c2wszbtakt3e > example_repo.car + # to clear the auth token env variable in current shell + unset ATP_AUTH_TOKEN - adenosine xrpc get app.bsky.getHomeFeed + # create a new session (login) for existing account + adenosine account login -u voltaire.test -p bogus + { + "did": "did:plc:yqtuksvatmmgngd5nkkw75hn", + "jwt": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJkaWQ6cGxjOnlxdHVrc3ZhdG1tZ25nZDVua2t3NzVobiIsImlhdCI6MTY2Njk5NjQxNX0.j2wcF1g9NxT_1AvYRiplNf_jtK6S81y3L38AkcBwOqY", + "name": "voltaire.test" + } + export ATP_AUTH_TOKEN=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJkaWQ6cGxjOnlxdHVrc3ZhdG1tZ25nZDVua2t3NzVobiIsImlhdCI6MTY2Njk5NjQxNX0.j2wcF1g9NxT_1AvYRiplNf_jtK6S81y3L38AkcBwOqY + +You could save the `ATP_HOST` and `ATP_AUTH_TOKEN` values in `~/.bashrc` so you +don't need to enter them every time. + +Now you can start posting and poking around: + + adenosine status -Example commands in bsky.app Lexicon: + adenosine bsky post "gruel again for breakfast" + { + "cid": "bafyreig2aqlsg4arslck64wbo2hnhe6k2a4z3z2sjfzh3uapv3a4zjld7e", + "uri": "at://did:plc:yqtuksvatmmgngd5nkkw75hn/app.bsky.post/3jg5zkr322c2a" + } - adenosine bsky feed + adenosine ls at://voltaire.test - adenosine bsky follow did:plc:mj35frj7bgu6c2wszbtakt3e - adenosine bsky profile +## Development and Contributions - adenosine bsky profile voltaire.test +Minimum Supported Rust Version (MSRV) is currently 1.61, using 2021 Rust +Edition. - adenosine get at://did:plc:mj35frj7bgu6c2wszbtakt3e/app.bsky.post/3jg4dqseut22a +Contributions, conversations, bug reports, and handwriten postcards are all +welcome. This is a low-effort hobby project so high-touch support, feature +requests, growth strategies, etc, probably will not be fielded. +The source code license is AGPLv3. Please acknowledge this in any initial +contributions, and also indicate whether and how you would like to be +acknowledged as a contributor (eg, a name and optionally a URL). diff --git a/adenosine-pds/README.md b/adenosine-pds/README.md index 77eae7f..91e6f5e 100644 --- a/adenosine-pds/README.md +++ b/adenosine-pds/README.md @@ -1,5 +1,98 @@ -Note: the `iroh-car` crate, which has not (yet) been published to crates.io, -has been vendored out of the `iroh` project in to a sub-module of this crate. -This is basically a verbatim copy, and the original copyright and license -applies to that code. +`adenosine-pds`: small-world atproto.com Personal Data Server +============================================================= + +This is a simple, enthusiast-grade AT Protocol +([atproto.com](https://atproto.com)) personal data server ("PDS") +implementation. It targets "small-world" uses cases of the protocol, for +example personal or organizational self-hosting. + +There is a [manpage](../extra/adenosine-pds.1.md) with usage and configuration +details. + +Features: + +- [x] com.atproto Lexicon implementation (not validated) +- [x] crude repo CAR file export (full history) +- [x] partial app.bsky Lexicon implementation (posts, follows) +- [x] crude internal did:plc generation +- [x] crude repo storage and manipulation +- [x] self-hosted did:web configuration +- [x] small-world registration and did:plc hosting configuration +- [ ] server-to-server interaction, at all (repo sync, etc) +- [ ] remote DID resolution (did:plc, did:web, etc) +- [ ] Lexicon schema validation of records/requests +- [ ] repo CAR file import: validation and app db update +- [ ] did:plc server interaction and chain validation +- [ ] complete app.bsky Lexicon implementation (notifications, likes, actors, etc) + +Two main deployment configurations are supported: + +- "fixed did:web accounts": accounts can only be registered at the CLI, and + did:web is used instead of did:plc. A did:web DID document is served from + `/.well-known/did.json` for matching domains, and a profile/feed is served + from the homepage when domain matches +- "domain wildcard registration": accounts can be registered via XRPC, limited + to handles under a specific hosting domain, and possibly requiring a secret + invite code. did:plc identifiers are generated locally. web views are served + from any domain, with registered handle domains being a profile/feed view + +## Quickstart + +TODO + +- generate a PDS-wide secret key, and store as env configuration variable +- set other config variables (see `--help` or manpage) + +## Deployment + +TODO: rewrite this + +- generate a PDS-wide secret key, and store as env configuration variable +- set other config variables (see `--help` or manpage) +- set up reverse proxy with SSL (eg, caddy or nginx or haproxy) +- run command from an appropriate local working directory (for databases) +- maybe set up a systemd unit file +- `adenosine-pds serve -vv` + + +## Implementation Details + +This is a Rust programming language project. + +Currently uses `rouille` as a (blocking) web framework; `rusqlite` for +(blocking) direct sqlite database interaction; `ipfs-sqlite-block-store` as a +(blocking) IPLD block store. + +A bunch of cleanup and refactoring would be useful: + +- cleaner ownership and referencing (reduce wasteful allocations) +- error handling (reduce `unwrap()` hacks) +- avoid mutex poisoning +- switch to async/await concurrency (replace or wrap blocking datastores) (eg, + `axum`, `sqlx`, `iroh` blockstore) + + +### Vendored Code + +The `iroh-car` crate, which has not (yet) been published to crates.io, has been +vendored out of the `iroh` project in to a sub-module of this crate. This is +basically a verbatim copy, and the original copyright and license applies to +that code. + +`ucan` support for the `p256` key type is implemented in this crate. Hoping to +get this upstreamed. + + +## Development and Contributions + +Minimum Supported Rust Version (MSRV) is currently 1.61, using 2021 Rust +Edition. + +Contributions, conversations, bug reports, and handwriten postcards are all +welcome. This is a low-effort hobby project so high-touch support, feature +requests, growth strategies, etc, probably will not be fielded. + +The source code license is AGPLv3. Please acknowledge this in any initial +contributions, and also indicate whether and how you would like to be +acknowledged as a contributor (eg, a name and optionally a URL). |