diff options
| author | Bryan Newbold <bnewbold@archive.org> | 2021-11-03 19:21:54 -0700 |
|---|---|---|
| committer | Bryan Newbold <bnewbold@archive.org> | 2021-11-03 19:21:57 -0700 |
| commit | 891299fd461b17c60fb48364cd5dca08c0711c32 (patch) | |
| tree | c079448b1e5bfbd5d0acdf2baadc52d05b91352d /notes/backfill_scalding_rewrite.txt | |
| parent | 848556a64d13955c2978bad352f2e2cd9edb62d0 (diff) | |
| download | sandcrawler-891299fd461b17c60fb48364cd5dca08c0711c32.tar.gz sandcrawler-891299fd461b17c60fb48364cd5dca08c0711c32.zip | |
IA (wayback): actually use an HTTP session for replay fetches
I am embarassed this wasn't actually the case already! Looks like I had
even instantiated a session but wasn't using it.
Hopefully this change, which adds extra retries and better backoff
behavior, will improve sandcrawler ingest throughput.
Diffstat (limited to 'notes/backfill_scalding_rewrite.txt')
0 files changed, 0 insertions, 0 deletions