diff options
| author | Bryan Newbold <bnewbold@archive.org> | 2020-01-02 18:10:23 -0800 |
|---|---|---|
| committer | Bryan Newbold <bnewbold@archive.org> | 2020-01-02 18:13:03 -0800 |
| commit | ddf56878e867e3556cb9c6cb6df6e60cff4e21fa (patch) | |
| tree | 27d244c4594c6f992ef9752e87b591fd246c63c0 | |
| parent | 629290b5713f4d33688fdbda614707d82898b0af (diff) | |
| download | sandcrawler-ddf56878e867e3556cb9c6cb6df6e60cff4e21fa.tar.gz sandcrawler-ddf56878e867e3556cb9c6cb6df6e60cff4e21fa.zip | |
minio version/compression notes
| -rw-r--r-- | minio/README.md | 11 |
1 files changed, 11 insertions, 0 deletions
diff --git a/minio/README.md b/minio/README.md index fd914f0..d8f1c69 100644 --- a/minio/README.md +++ b/minio/README.md @@ -4,6 +4,14 @@ documents, addressed by the sha1 of the PDF file the XML was extracted from. Note that on the backend minio is just storing objects as files on disk. +## Deploying minio Server + +It seems to be important to use a version of minio from at least December 2019 +era for on-disk compression to actually work. + +Currently install minio (and mc, the minio client) in prod by simply +downloading the binaries and calling from systemd. + ## Buckets and Directories Hosts and buckets: @@ -61,3 +69,6 @@ Make a prefix within a bucket world-readable like: mc policy set download cluster/unpaywall/grobid +## Config + + mc admin config set aitio compression extensions=.txt,.log,.csv,.json,.tsv,.pdf,.xml mime_types=text/csv,text/plain,application/json,application/xml,application/octet-stream,application/tei+xml |