diff options
| author | Bryan Newbold <bnewbold@archive.org> | 2020-06-29 13:19:17 -0700 |
|---|---|---|
| committer | Bryan Newbold <bnewbold@archive.org> | 2020-06-29 13:19:22 -0700 |
| commit | 800860ecd25346ff4a638e9d42fa905396b8fa1b (patch) | |
| tree | 736e032c1d412d2cfcc0b9807e2ce55bde35ba8c /python/sandcrawler_worker.py | |
| parent | c216a9cdecd85db8296e6499593244686c430d8c (diff) | |
| download | sandcrawler-800860ecd25346ff4a638e9d42fa905396b8fa1b.tar.gz sandcrawler-800860ecd25346ff4a638e9d42fa905396b8fa1b.zip | |
customize timeout per worker; 120sec for pdf-extract
This is a stab-in-the-dark attempt to resolve long timeouts with this
worker in prod.
Diffstat (limited to 'python/sandcrawler_worker.py')
| -rwxr-xr-x | python/sandcrawler_worker.py | 1 |
1 files changed, 1 insertions, 0 deletions
diff --git a/python/sandcrawler_worker.py b/python/sandcrawler_worker.py index 0fd0194..833b9c4 100755 --- a/python/sandcrawler_worker.py +++ b/python/sandcrawler_worker.py @@ -75,6 +75,7 @@ def run_pdf_extract(args): consume_topic=consume_topic, group="pdf-extract", batch_size=1, + push_timeout_sec=120, ) pusher.run() |