notes/bulk_edits/2020-08-05_chocula.md


1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17


Using `journal-metadata-bot` and `chocula_fatcat_export.2020-07-31.json` export.

Start small:

    head -n100 /srv/fatcat/datasets/chocula_fatcat_export.2020-07-31.json | ./fatcat_import.py chocula --do-updates -
    => Counter({'total': 100, 'update': 67, 'exists': 25, 'exists-skip-update': 24, 'insert': 8, 'exists-by-issnl': 1, 'skip': 0})

Full batch:

    time cat /srv/fatcat/datasets/chocula_fatcat_export.2020-07-31.json | ./fatcat_import.py chocula --do-updates -

    Counter({'total': 164950, 'update': 112074, 'exists': 37243, 'exists-skip-update': 35862, 'insert': 15633, 'exists-by-issnl': 1381, 'skip': 0})

    real    16m9.779s
    user    6m26.324s
    sys     0m16.088s