diff options
| author | Bryan Newbold <bnewbold@archive.org> | 2021-04-27 17:05:56 -0700 |
|---|---|---|
| committer | Bryan Newbold <bnewbold@archive.org> | 2021-04-27 17:05:59 -0700 |
| commit | 3e6ec2fbf71518397353d7ea25bb474ba2278a58 (patch) | |
| tree | 23c057cc7552521f9c5bd7bd5531135c525d90a4 /notes/url_pattern_heuristic_backfill.txt | |
| parent | 433da53798b095188d3112aa3f4b509d92a3adec (diff) | |
| download | sandcrawler-3e6ec2fbf71518397353d7ea25bb474ba2278a58.tar.gz sandcrawler-3e6ec2fbf71518397353d7ea25bb474ba2278a58.zip | |
ingest: cap max body size to ~128 MByte
Should resolve 'magic' OOM errors in production.
Diffstat (limited to 'notes/url_pattern_heuristic_backfill.txt')
0 files changed, 0 insertions, 0 deletions