summaryrefslogtreecommitdiffstats
path: root/extra/partition_dumps
diff options
context:
space:
mode:
authorBryan Newbold <bnewbold@robocracy.org>2018-09-25 15:36:27 -0700
committerBryan Newbold <bnewbold@robocracy.org>2018-09-25 15:36:27 -0700
commitf41ba2dc9ab8e5d4c9497d8845ab1c16e08ccd05 (patch)
tree0548e03ec0a777a97a601edf4ba5d38860e9016f /extra/partition_dumps
parentbf4bb40692c45d218f03443cf09529c459b1ac90 (diff)
downloadfatcat-f41ba2dc9ab8e5d4c9497d8845ab1c16e08ccd05.tar.gz
fatcat-f41ba2dc9ab8e5d4c9497d8845ab1c16e08ccd05.zip
better default file names
Diffstat (limited to 'extra/partition_dumps')
-rw-r--r--extra/partition_dumps/README.md14
1 files changed, 7 insertions, 7 deletions
diff --git a/extra/partition_dumps/README.md b/extra/partition_dumps/README.md
index 2e26a41b..5e42ff48 100644
--- a/extra/partition_dumps/README.md
+++ b/extra/partition_dumps/README.md
@@ -5,9 +5,9 @@ journal/container.
Example parititoning a sample by release type:
- cat release_dump_expanded_sample.json | jq .release_type -r > release_dump_expanded_sample.release_type
- cat release_dump_expanded_sample.release_type | sort | uniq -c | sort -nr > release_dump_expanded_sample.release_type.counts
- cat release_dump_expanded_sample.json | paste release_dump_expanded_sample.release_type - | sort > out
+ cat release_export_expanded_sample.json | jq .release_type -r > release_export_expanded_sample.release_type
+ cat release_export_expanded_sample.release_type | sort | uniq -c | sort -nr > release_export_expanded_sample.release_type.counts
+ cat release_export_expanded_sample.json | paste release_export_expanded_sample.release_type - | sort > out
More production-y example using ISSN-L:
@@ -15,11 +15,11 @@ More production-y example using ISSN-L:
rm -rf ./partitioned
# it's a pretty huge sort, will need 300+ GB scratch space? this might not scale.
- zcat release_dump_expanded.json.gz | jq .container.issnl -r > release_dump_expanded.issnl
- zcat release_dump_expanded.json.gz | paste release_dump_expanded.issnl - | sort | ./partition_script.py
+ zcat release_export_expanded.json.gz | jq .container.issnl -r > release_export_expanded.issnl
+ zcat release_export_expanded.json.gz | paste release_export_expanded.issnl - | sort | ./partition_script.py
# for verification/stats
- cat release_dump_expanded.issnl | sort | uniq -c | sort -nr > release_dump_expanded.issnl.counts
+ cat release_export_expanded.issnl | sort | uniq -c | sort -nr > release_export_expanded.issnl.counts
# cleanup
- rm release_dump_expanded.issnl
+ rm release_export_expanded.issnl