| Commit message (Collapse) | Author | Age | Files | Lines | ||
|---|---|---|---|---|---|---|
| ... | ||||||
| * | progress on extractor | Bryan Newbold | 2018-04-05 | 3 | -56/+93 | |
| | | ||||||
| * | improve test coverage | Bryan Newbold | 2018-04-05 | 6 | -6/+39 | |
| | | ||||||
| * | test coverage info | Bryan Newbold | 2018-04-05 | 4 | -7/+67 | |
| | | ||||||
| * | README/TODO updates | Bryan Newbold | 2018-04-04 | 3 | -9/+20 | |
| | | ||||||
| * | refactor out some common code | Bryan Newbold | 2018-04-04 | 5 | -184/+133 | |
| | | ||||||
| * | extraction -> mapreduce | Bryan Newbold | 2018-04-04 | 14 | -0/+0 | |
| | | ||||||
| * | merge backfill into extraction directory | Bryan Newbold | 2018-04-04 | 11 | -653/+27 | |
| | | ||||||
| * | pep8 | Bryan Newbold | 2018-04-04 | 2 | -3/+3 | |
| | | ||||||
| * | testing stuff as dev deps | Bryan Newbold | 2018-04-04 | 2 | -73/+109 | |
| | | ||||||
| * | more testing deps | Bryan Newbold | 2018-04-04 | 2 | -8/+133 | |
| | | ||||||
| * | trivial whitespace | Bryan Newbold | 2018-04-04 | 2 | -1/+2 | |
| | | ||||||
| * | more TODO | Bryan Newbold | 2018-04-04 | 2 | -0/+17 | |
| | | ||||||
| * | more WIP on extractor | Bryan Newbold | 2018-04-04 | 5 | -52/+427 | |
| | | ||||||
| * | add example XML output (open access) | Bryan Newbold | 2018-04-03 | 1 | -0/+2004 | |
| | | ||||||
| * | WIP on extractor-with-mrjob | Bryan Newbold | 2018-04-03 | 4 | -0/+954 | |
| | | ||||||
| * | fix very important typo | Bryan Newbold | 2018-04-03 | 1 | -1/+1 | |
| | | ||||||
| * | shift docs around a bit | Bryan Newbold | 2018-04-03 | 2 | -9/+12 | |
| | | ||||||
| * | actually running hadoop job on cluster | Bryan Newbold | 2018-04-03 | 2 | -0/+18 | |
| | | ||||||
| * | fix silly bugs in backfiller (need more tests) | Bryan Newbold | 2018-04-03 | 1 | -3/+4 | |
| | | ||||||
| * | add setuptools (can probably remove) | Bryan Newbold | 2018-04-03 | 2 | -7/+8 | |
| | | ||||||
| * | heritrix expects ints, not strings, for numbers | Bryan Newbold | 2018-04-02 | 1 | -7/+7 | |
| | | ||||||
| * | backfill: sha1 prefix, cluster example | Bryan Newbold | 2018-03-30 | 3 | -8/+19 | |
| | | ||||||
| * | clean up backfill code/tests | Bryan Newbold | 2018-03-30 | 2 | -24/+42 | |
| | | ||||||
| * | refactor backfill for mrjob | Bryan Newbold | 2018-03-30 | 4 | -64/+145 | |
| | | ||||||
| * | pytest helpers | Bryan Newbold | 2018-03-30 | 4 | -32/+564 | |
| | | ||||||
| * | clean up pig test stuff | Bryan Newbold | 2018-03-30 | 6 | -62/+71 | |
| | | ||||||
| * | renames | Bryan Newbold | 2018-03-30 | 4 | -0/+129 | |
| | | ||||||
| * | basically working pig test | Bryan Newbold | 2018-03-29 | 5 | -23/+32 | |
| | | ||||||
| * | progress on pig tests | Bryan Newbold | 2018-03-29 | 8 | -10/+127 | |
| | | ||||||
| * | import WIP on pig test setup | Bryan Newbold | 2018-03-29 | 6 | -0/+156 | |
| | | ||||||
| * | WIP on cdx backfill | Bryan Newbold | 2018-03-29 | 3 | -0/+265 | |
| | | ||||||
| * | move to top level | Bryan Newbold | 2018-03-29 | 3 | -0/+0 | |
| | | ||||||
| * | sandcrawler | Bryan Newbold | 2018-03-29 | 1 | -2/+9 | |
| | | ||||||
| * | no venvs | Bryan Newbold | 2018-03-29 | 1 | -0/+1 | |
| | | ||||||
| * | import vinay's cdx-record-pipeline | Bryan Newbold | 2018-03-29 | 3 | -0/+103 | |
| | | ||||||
| * | init repo | Bryan Newbold | 2018-03-29 | 2 | -0/+30 | |
 |
index : sandcrawler | |
| [no description] |
| aboutsummaryrefslogtreecommitdiffstats |