diff options
| author | Bryan Newbold <bnewbold@archive.org> | 2020-02-02 21:59:14 -0800 |
|---|---|---|
| committer | Bryan Newbold <bnewbold@archive.org> | 2020-02-02 21:59:16 -0800 |
| commit | 2a5f6a25123d6b725327de844da38df735b04d3f (patch) | |
| tree | 27c2869ff522f74eff47f189c97bfcbd97eb9652 /python/ingest_file.py | |
| parent | 6a29e396da2289e6524097ef906b63358eb3cfec (diff) | |
| download | sandcrawler-2a5f6a25123d6b725327de844da38df735b04d3f.tar.gz sandcrawler-2a5f6a25123d6b725327de844da38df735b04d3f.zip | |
wayback: try to resolve HTTPException due to many HTTP headers
This is withing GWB wayback code. Trying two things:
- bump default max headers from 100 to 1000 in the (global?) http.client
module itself. I didn't think through whether we would expect this to
actually work
- catch the exception, record it, move on
Diffstat (limited to 'python/ingest_file.py')
0 files changed, 0 insertions, 0 deletions