Commit message (Collapse) | Author | Age | Files | Lines | |
---|---|---|---|---|---|
* | wrap up basic (locally testable) ingest refactor | Bryan Newbold | 2020-01-09 | 1 | -4/+48 |
| | |||||
* | basic elife+plos extraction tests | Bryan Newbold | 2020-01-09 | 3 | -0/+4842 |
| | | | | | Ripped out some HTML, but these could have been minimized even further to keep repository from growing large. | ||||
* | fix grobid test (ISO-8859-1 encoding) | Bryan Newbold | 2020-01-09 | 1 | -6/+4 |
| | | | | Also changes for wayback refactor | ||||
* | fix grobid tests for new wayback refactors | Bryan Newbold | 2020-01-09 | 2 | -12/+14 |
| | |||||
* | more wayback and SPN tests and fixes | Bryan Newbold | 2020-01-09 | 2 | -13/+67 |
| | |||||
* | refactor CdxApiClient, add tests | Bryan Newbold | 2020-01-08 | 1 | -0/+110 |
| | | | | | | - always use auth token and get full CDX rows - simplify to "fetch" (exact url/dt match) and "lookup_best" methods - all redirect stuff will be moved to a higher level | ||||
* | refactor SavePaperNowClient and add test | Bryan Newbold | 2020-01-07 | 1 | -0/+160 |
| | | | | | | - response as a namedtuple - "remote" errors (aka, SPN API was HTTP 200 but returned error) aren't an exception | ||||
* | teixml2json test update for skipping null JSON keys | Bryan Newbold | 2020-01-02 | 1 | -10/+1 |
| | |||||
* | grobid2json: language_code | Bryan Newbold | 2019-10-04 | 1 | -1/+2 |
| | |||||
* | python tests for pusher classes | Bryan Newbold | 2019-10-02 | 2 | -0/+28 |
| | |||||
* | add tests for affiliation extraction | Bryan Newbold | 2019-10-02 | 2 | -1/+25 |
| | |||||
* | lots of grobid tool implementation (still WIP) | Bryan Newbold | 2019-09-26 | 2 | -7/+29 |
| | |||||
* | test of GROBID client | Bryan Newbold | 2019-09-25 | 1 | -0/+53 |
| | |||||
* | refactor old python hadoop code into new directory | Bryan Newbold | 2019-09-25 | 4 | -591/+0 |
| | |||||
* | re-write parse_cdx_line for sandcrawler lib | Bryan Newbold | 2019-09-25 | 1 | -1/+31 |
| | |||||
* | fix test grobid2json test | Bryan Newbold | 2019-09-25 | 1 | -1/+4 |
| | | | | For new extra fields | ||||
* | start refactoring sandcrawler python common code | Bryan Newbold | 2019-09-23 | 2 | -0/+41 |
| | |||||
* | update grobid2json to include given_name/surname | Bryan Newbold | 2019-05-13 | 1 | -3/+3 |
| | |||||
* | python test fixes | Bryan Newbold | 2019-02-21 | 1 | -1/+1 |
| | |||||
* | fix ungrobid extraction tests | Bryan Newbold | 2018-11-22 | 1 | -2/+4 |
| | |||||
* | longtail grobid metadata parse/filter WIP | Bryan Newbold | 2018-09-22 | 1 | -0/+5 |
| | |||||
* | WIP: ungrobided doesn't inherit (copypasta) | Bryan Newbold | 2018-08-25 | 1 | -4/+4 |
| | |||||
* | ungrobided: example real output | Bryan Newbold | 2018-08-25 | 1 | -0/+20 |
| | |||||
* | ungrobided: add real results to tests | Bryan Newbold | 2018-08-25 | 1 | -1/+51 |
| | |||||
* | python extraction_ungrobided job | Bryan Newbold | 2018-08-24 | 1 | -0/+126 |
| | |||||
* | rename ./mapreduce to ./python | Bryan Newbold | 2018-08-24 | 8 | -0/+2632 |