| Commit message (Collapse) | Author | Age | Files | Lines | |
|---|---|---|---|---|---|
| * | remove deprecated kafka_grobid.py worker | Bryan Newbold | 2020-05-26 | 1 | -331/+0 |
| | | | | | | | All use of pykafka was refactored to use the confluent library some time ago. And all kafka workers have been using the newer sandcrawler style worker for some time. | ||||
| * | refactor: use print(..., file=sys.stderr) | Bryan Newbold | 2019-12-18 | 1 | -2/+2 |
| | | | | | Should use logging soon, but this seems more idiomatic in the meanwhile. | ||||
| * | refactor: sort keys in JSON output | Bryan Newbold | 2019-12-18 | 1 | -1/+1 |
| | | | | | This makes debugging by tailing Kafka topics a lot more readable | ||||
| * | refactor: improve argparse usage | Bryan Newbold | 2019-12-18 | 1 | -1/+2 |
| | | | | | | use ArgumentDefaultsHelpFormatter and add help messages to all sub-commands | ||||
| * | note that kafka_grobid.py is deprecated | Bryan Newbold | 2019-11-13 | 1 | -0/+3 |
| | | |||||
| * | python test fixes | Bryan Newbold | 2019-02-21 | 1 | -0/+1 |
| | | |||||
| * | backport GWB fetch improvements to extraction/kafka workers | Bryan Newbold | 2019-02-21 | 1 | -4/+8 |
| | | | | | *Really* need to refactor out these common methods into a base class. | ||||
| * | ah, right, it's more like extract/3sec, not 30sec | Bryan Newbold | 2018-12-03 | 1 | -4/+4 |
| | | |||||
| * | tweak grobid worker producer settings | Bryan Newbold | 2018-12-03 | 1 | -2/+2 |
| | | | | | | Python CPU utilization shot way up; this is an attempt to bring it back down. | ||||
| * | tweak kafka config significantly | Bryan Newbold | 2018-12-03 | 1 | -3/+16 |
| | | |||||
| * | more sentry tags when extracting | Bryan Newbold | 2018-12-03 | 1 | -1/+6 |
| | | |||||
| * | improvements to Kafka GROBID worker logging | Bryan Newbold | 2018-12-03 | 1 | -5/+11 |
| | | |||||
| * | fix error var typo | Bryan Newbold | 2018-11-27 | 1 | -1/+1 |
| | | |||||
| * | catch more wayback error types | Bryan Newbold | 2018-11-26 | 1 | -1/+11 |
| | | |||||
| * | better default consumergroup name | Bryan Newbold | 2018-11-21 | 1 | -1/+1 |
| | | |||||
| * | fix kafka grobid command line topic parsing | Bryan Newbold | 2018-11-21 | 1 | -2/+2 |
| | | |||||
| * | kafka_grobid fixes and hbase WIP | Bryan Newbold | 2018-11-21 | 1 | -2/+6 |
| | | |||||
| * | small kafka_grobid tweaks | Bryan Newbold | 2018-11-21 | 1 | -1/+2 |
| | | |||||
| * | kafka_grobid tweaks for deployment; delay insert decision | Bryan Newbold | 2018-11-21 | 1 | -21/+9 |
| | | |||||
| * | initial work on kafka_grobid worker | Bryan Newbold | 2018-11-20 | 1 | -0/+295 |
 |
index : sandcrawler | |
| [no description] |
| aboutsummaryrefslogtreecommitdiffstats |