aboutsummaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
authorBryan Newbold <bnewbold@archive.org>2020-01-02 18:10:23 -0800
committerBryan Newbold <bnewbold@archive.org>2020-01-02 18:13:03 -0800
commitddf56878e867e3556cb9c6cb6df6e60cff4e21fa (patch)
tree27d244c4594c6f992ef9752e87b591fd246c63c0
parent629290b5713f4d33688fdbda614707d82898b0af (diff)
downloadsandcrawler-ddf56878e867e3556cb9c6cb6df6e60cff4e21fa.tar.gz
sandcrawler-ddf56878e867e3556cb9c6cb6df6e60cff4e21fa.zip
minio version/compression notes
-rw-r--r--minio/README.md11
1 files changed, 11 insertions, 0 deletions
diff --git a/minio/README.md b/minio/README.md
index fd914f0..d8f1c69 100644
--- a/minio/README.md
+++ b/minio/README.md
@@ -4,6 +4,14 @@ documents, addressed by the sha1 of the PDF file the XML was extracted from.
Note that on the backend minio is just storing objects as files on disk.
+## Deploying minio Server
+
+It seems to be important to use a version of minio from at least December 2019
+era for on-disk compression to actually work.
+
+Currently install minio (and mc, the minio client) in prod by simply
+downloading the binaries and calling from systemd.
+
## Buckets and Directories
Hosts and buckets:
@@ -61,3 +69,6 @@ Make a prefix within a bucket world-readable like:
mc policy set download cluster/unpaywall/grobid
+## Config
+
+ mc admin config set aitio compression extensions=.txt,.log,.csv,.json,.tsv,.pdf,.xml mime_types=text/csv,text/plain,application/json,application/xml,application/octet-stream,application/tei+xml