1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
|
Rust implementation of fatcat API server. Commands include:
- `fatcatd`: the API server itself
- `fatcat-auth`: privileged command to manage authentication keys, tokens, and
accounts. Useful to generate admin accounts, new signing keys, etc.
- `fatcat-export`: high-speed JSON export tool, which talks directly to the
database (instead of going through the API).
The `fatcat-api-spec` crate is generated from the openapi/swagger spec and
contains Rust models, response types, and endpoint definitions (but not
implementations).
The SQL database schema (and migrations) are under `./migrations/`.
## Development
You need the following dependencies installed locally to build, run tests, and
do development work:
- rust stable, 2018 edition, 1.34+ (eg, via "rustup", includes cargo tool)
- diesel (`cargo install diesel_cli`)
- postgres (compatible with 9.6+; we run 11.x in production)
- postgres libs (debian/ubuntu: `libsqlite3-dev libpq-dev`)
- libsodium library and development headers (debian/ubuntu: `libsodium-dev`)
- OpenSSL 1.0 (not compatible with 1.1; see "OpenSSL 1.0" section below)
We need to create a new `fatcat` postgres user and database to run tests and
develop with. On debian/ubuntu, a UNIX account named `postgres` is
automatically created with control over the database, so we'll run setup
commands from that user. To create the database account:
sudo su - postgres
# this command creates a PostgreSQL user, not a UNIX/system user
createuser -s fatcat -P
# switch back to your regular system user
exit
Copy `./example.env` to `./.env` and update the `DATABASE_URL` and
`TEST_DATABASE_URL` lines with the password you set above.
As your regular user, use `diesel` to create and initialize the `fatcat`
database (`diesel` and the fatcat tools will automatically use postgresql
credentials from the `.env` file):
diesel database reset
Build and run the API server:
cargo run --bin fatcatd
Run tests:
cargo test -- --test-threads 1
Note that most "integration" level tests are written in python and run by
`pytest`; see `../python/README.md`.
See `HACKING` for some more advanced tips and commands.
### OpenSSL 1.0
Ubuntu bionic (18.04) and later ship with OpenSSL 1.1, you can install OpenSSL
1.0 with:
# note that libssl1.0-dev conflicts with libssl-dev
sudo apt install openssl1.0 libssl1.0-dev
And then build with:
cargo clean
OPENSSL_LIB_DIR="/usr/lib/x86_64-linux-gnu" OPENSSL_INCLUDE_DIR="/usr/include/openssl" cargo build --release
There is a [github tracking issue](https://github.com/internetarchive/fatcat/issues/42)
for this dependency problem.
## Configuration
All configuration goes through environment variables, the notable ones being:
- `DATABASE_URL`: postgres connection details (username, password, host, and database)
- `TEST_DATABASE_URL`: used when running `cargo test`
- `AUTH_LOCATION`: the domain authentication tokens should be valid over
- `AUTH_KEY_IDENT`: a unique name for the primary auth signing key (used to
find the correct key after key rotation has occured)
- `AUTH_SECRET_KEY`: base64-encoded secret key used to both sign and verify
authentication tokens (symmetric encryption)
- `AUTH_ALT_KEYS`: additional ident/key pairs that can be used to verify tokens
(to enable key rotation). Syntax is like `<ident1>:<key1>,<ident2>:key2,...`.
To setup authentication with a new secret authentication key, run:
cargo run --bin fatcat-auth create-key
then copy the last line as `AUTH_SECRET_KEY` in `.env`, and update
`AUTH_KEY_IDENT` with a unique name for this new key (eg, including the date).
|
