index
:
sandcrawler
bnewbold-args
bnewbold-backfill
bnewbold-persist-grobid-errors
bnewbold-refactor-loggging
master
trawler
[no description]
about
summary
refs
log
tree
commit
diff
stats
log msg
author
committer
range
path:
root
/
python
/
sandcrawler
/
workers.py
Commit message (
Expand
)
Author
Age
Files
Lines
*
improvements to reliability from prod testing
Bryan Newbold
2020-02-03
1
-2
/
+9
*
hack-y backoff ingest attempt
Bryan Newbold
2020-02-03
1
-1
/
+15
*
worker kafka setting tweaks
Bryan Newbold
2020-01-28
1
-2
/
+4
*
workers: yes, poll is necessary
Bryan Newbold
2020-01-28
1
-1
/
+1
*
fix kafka worker partition-specific error
Bryan Newbold
2020-01-28
1
-1
/
+1
*
have JsonLinePusher continue on JSON decode errors (but count)
Bryan Newbold
2020-01-02
1
-1
/
+5
*
refactor: use print(..., file=sys.stderr)
Bryan Newbold
2019-12-18
1
-20
/
+22
*
CI: make some jobs manual
Bryan Newbold
2019-11-15
1
-0
/
+2
*
bump kafka max poll interval for consumers
Bryan Newbold
2019-11-14
1
-2
/
+2
*
update ingest-file batch size to 1
Bryan Newbold
2019-11-14
1
-3
/
+3
*
refactor consume_topic name out of make_kafka_consumer()
Bryan Newbold
2019-11-13
1
-5
/
+5
*
workers: better generic batch-size arg handling
Bryan Newbold
2019-10-03
1
-0
/
+6
*
more counts and bugfixes in grobid_tool
Bryan Newbold
2019-09-26
1
-0
/
+6
*
off-by-one error in batch sizes
Bryan Newbold
2019-09-26
1
-1
/
+1
*
lots of grobid tool implementation (still WIP)
Bryan Newbold
2019-09-26
1
-0
/
+419