update minio README

1 files changed, 42 insertions, 10 deletions

diff --git a/minio/README.md b/minio/README.md
index 3ce0f95..fd914f0 100644
--- a/minio/README.md
+++ b/minio/README.md
@@ -4,21 +4,52 @@ documents, addressed by the sha1 of the PDF file the XML was extracted from.
 
 Note that on the backend minio is just storing objects as files on disk.
 
-## Buckets
+## Buckets and Directories
 
-Notable buckets, and structure/naming convention:
+Hosts and buckets:
 
-    grobid/
-        2c/0d/2c0daa9307887a27054d4d1f137514b0fa6c6b2d.tei.xml
-        SHA1 (lower-case hex) of PDF that XML was extracted from
-    unpaywall/grobid/
-        2c/0d/2c0daa9307887a27054d4d1f137514b0fa6c6b2d.tei.xml
+    localhost:sandcrawler-dev
+        create locally for development (see below)
+
+    cluster:sandcrawler
+        main sandcrawler storage bucket, for GROBID output and other derivatives.
+        Note it isn't "sandcrawler-prod", for backwards compatibility reasons.
+
+    cluster:sandcrawler-qa
+        for, eg, testing on cluster servers
+
+    cluster:unpaywall
+        subset of sandcrawler content crawled due to unpaywall URLs;
+        potentially made publicly accessible
+
+Directory structure within sandcrawler buckets:
+
+    grobid/2c/0d/2c0daa9307887a27054d4d1f137514b0fa6c6b2d.tei.xml
         SHA1 (lower-case hex) of PDF that XML was extracted from
-        (mirror of /grobid/ for which we crawled for unpaywall and made publicly accessible)
 
 Create new buckets like:
 
-    mc mb sandcrawler/grobid
+    mc mb cluster/sandcrawler-qa
+
+## Development
+
+Run minio server locally, with non-persisted data:
+
+    docker run -p 9000:9000 minio/minio server /data
+
+Credentials are `minioadmin:minioadmin`. Install `mc` client utility, and
+configure:
+
+    mc config host add localhost http://localhost:9000 minioadmin minioadmin
+
+Then create dev bucket:
+
+    mc mb --ignore-existing localhost/sandcrawler-dev
+
+A common "gotcha" with `mc` command is that it will first look for a local
+folder/directory with same name as the configured remote host, so make sure
+there isn't a `./localhost` folder.
+
 
 ## Users
 
@@ -28,4 +59,5 @@ Create a new readonly user like:
 
 Make a prefix within a bucket world-readable like:
 
-    mc policy set download sandcrawler/unpaywall/grobid
+    mc policy set download cluster/unpaywall/grobid
+