fatcat/python/fatcat_tools/harvest, branch confluent-kafka[no description]https://git.bnewbold.net/fatcat/atom?h=confluent-kafka2019-04-09T04:51:36Zfix missing KafkaException harvester imports2019-04-09T04:51:36ZBryan Newboldbnewbold@robocracy.org2019-04-09T04:51:36Zurn:sha1:95943072da4afd1611b790d9900048100545a803small kafka tweaks for robustness2019-04-09T04:10:45ZBryan Newboldbnewbold@robocracy.org2019-04-09T04:10:45Zurn:sha1:aac6727d2fdf59e09c1f42da19a9a894304acc00bump max message size to ~20 MBytes2019-04-08T22:04:50ZBryan Newboldbnewbold@robocracy.org2019-04-08T22:04:50Zurn:sha1:d43ee5f73e1f7d7f6317d1e7dfb1ddd0e510a353fixes to confluent-kafka harvesters2019-04-08T21:58:20ZBryan Newboldbnewbold@robocracy.org2019-04-08T21:58:20Zurn:sha1:953886e6e42294ca81583e9b2c4e2fa16343a86efirst draft harvesters using confluent-kafka2019-04-07T04:15:48ZBryan Newboldbnewbold@robocracy.org2019-04-07T04:15:48Zurn:sha1:8656ddf280aee3af459c3066679c534616fdca93increase default harvest window to 14 days2019-04-01T21:50:24ZBryan Newboldbnewbold@robocracy.org2019-04-01T21:50:24Zurn:sha1:cff3906f2ea6176d6395fb7eb1830dde4a096522HACK: force pylint to ignore urllib3 Retry import2019-03-15T23:53:56ZBryan Newboldbnewbold@robocracy.org2019-03-15T23:17:51Zurn:sha1:4f5d992bd75823e03ef64a77104b4ddee08aaf12
As the code comment mentions, not sure why pylint throws this error.
requests and urllib3 are recent, and this code runs fine in tests and
QA, and pylint is running (in CI) within pipenv.
MEDLINE/Pubmed note2019-03-15T22:16:57ZBryan Newboldbnewbold@robocracy.org2019-03-15T22:16:50Zurn:sha1:d0772cc01462d532d0cd82cb0c6dd859f4c4e74d
Also, arXivRaw, not arXiv (though see WIP on more-importers branch)
fix harvester session.get() params2019-03-06T19:47:21ZBryan Newboldbnewbold@robocracy.org2019-03-06T19:47:21Zurn:sha1:7da130abec5ce9a8edbe6d2a4ef4c8be60e16a0dretry/backoff for Crossref harvester2019-03-06T19:39:36ZBryan Newboldbnewbold@robocracy.org2019-03-06T19:39:36Zurn:sha1:5dc322c93eb3b92324c4f947697d2a2c69749040