aboutsummaryrefslogtreecommitdiffstats
path: root/python/scripts/import_grobid_metadata.py
diff options
context:
space:
mode:
authorBryan Newbold <bnewbold@archive.org>2022-04-26 15:25:20 -0700
committerBryan Newbold <bnewbold@archive.org>2022-04-26 15:25:23 -0700
commit8f9240fed272367669b535b1334e280c588a1791 (patch)
treef2eaf2c0ff81290b482dba31ef1df650dac3b23d /python/scripts/import_grobid_metadata.py
parentf8f41070adfd4dc4856d96f618ee96fe8c411458 (diff)
downloadsandcrawler-8f9240fed272367669b535b1334e280c588a1791.tar.gz
sandcrawler-8f9240fed272367669b535b1334e280c588a1791.zip
SPNv2: several fixes for prod throughput
Most importantly, for some API flags, if the value is not true-thy, do not set the flag at all. Setting any flag was resulting in screenshots and outlinks actually getting created/captured, which was a huge slowdown. Also, check per-user SPNv2 slots available, using API, before requesting an actual capture.
Diffstat (limited to 'python/scripts/import_grobid_metadata.py')
0 files changed, 0 insertions, 0 deletions