diff options
| author | Bryan Newbold <bnewbold@archive.org> | 2022-04-26 15:25:20 -0700 |
|---|---|---|
| committer | Bryan Newbold <bnewbold@archive.org> | 2022-04-26 15:25:23 -0700 |
| commit | 8f9240fed272367669b535b1334e280c588a1791 (patch) | |
| tree | f2eaf2c0ff81290b482dba31ef1df650dac3b23d /python_hadoop/extraction_ungrobided.py | |
| parent | f8f41070adfd4dc4856d96f618ee96fe8c411458 (diff) | |
| download | sandcrawler-8f9240fed272367669b535b1334e280c588a1791.tar.gz sandcrawler-8f9240fed272367669b535b1334e280c588a1791.zip | |
SPNv2: several fixes for prod throughput
Most importantly, for some API flags, if the value is not true-thy, do
not set the flag at all. Setting any flag was resulting in screenshots
and outlinks actually getting created/captured, which was a huge
slowdown.
Also, check per-user SPNv2 slots available, using API, before requesting
an actual capture.
Diffstat (limited to 'python_hadoop/extraction_ungrobided.py')
0 files changed, 0 insertions, 0 deletions