| Commit message (Collapse) | Author | Age | Files | Lines | |
|---|---|---|---|---|---|
| * | hack scorejob variant with extra context joined in | Bryan Newbold | 2018-09-12 | 1 | -0/+262 |
| | | | | | This is to allow pushes to fatcat. There is probably a better way... | ||||
| * | blacklist -> denylist | Bryan Newbold | 2018-09-05 | 1 | -1/+1 |
| | | |||||
| * | changed style of ScoreJobTest.bundle | Ellen Spertus | 2018-09-04 | 1 | -14/+10 |
| | | |||||
| * | minor style improvement | Ellen Spertus | 2018-09-04 | 1 | -2/+2 |
| | | |||||
| * | fixed tests after replacing NoSlug with None | Ellen Spertus | 2018-09-04 | 4 | -77/+85 |
| | | |||||
| * | make similarity score case-insensitive | Bryan Newbold | 2018-08-27 | 1 | -0/+8 |
| | | |||||
| * | basic crossref subtitle concatination support | Bryan Newbold | 2018-08-27 | 1 | -0/+18 |
| | | |||||
| * | more special characters to strip | Bryan Newbold | 2018-08-27 | 1 | -1/+1 |
| | | |||||
| * | rename DumpUnGrobidedJob | Bryan Newbold | 2018-08-24 | 1 | -4/+4 |
| | | |||||
| * | scalding: UnGrobidedDumpJob | Bryan Newbold | 2018-08-24 | 1 | -0/+72 |
| | | |||||
| * | add a content-type filter for crossref works | Bryan Newbold | 2018-08-23 | 1 | -0/+9 |
| | | |||||
| * | require crossref works to have at least one author (for matching) | Bryan Newbold | 2018-08-23 | 1 | -0/+6 |
| | | |||||
| * | author parsing (and year, for crossref) | Bryan Newbold | 2018-08-23 | 2 | -1/+6 |
| | | |||||
| * | set a minimum slug size (8 chars) | Bryan Newbold | 2018-08-23 | 3 | -14/+26 |
| | | |||||
| * | Fixed style violations. | Ellen Spertus | 2018-08-22 | 1 | -2/+1 |
| | | |||||
| * | Added ScoreJob test for title-length filtering. | Ellen Spertus | 2018-08-22 | 1 | -5/+13 |
| | | |||||
| * | Merge branch 'master' into ellen-length-filtering | Ellen Spertus | 2018-08-22 | 1 | -1/+1 |
| |\ | |||||
| | * | add more punctuation characters to slug filter | Bryan Newbold | 2018-08-22 | 1 | -1/+1 |
| | | | |||||
| * | | Added title-length filtering to CrossrefScorable. | Ellen Spertus | 2018-08-22 | 1 | -2/+34 |
| | | | |||||
| * | | Added more tests of GrobidScorable.keepRecord | Ellen Spertus | 2018-08-22 | 1 | -0/+5 |
| | | | |||||
| * | | Added title length filtering to GrobidScorable | Ellen Spertus | 2018-08-22 | 1 | -2/+29 |
| |/ | |||||
| * | remove slug-blacklist conservative test | Bryan Newbold | 2018-08-21 | 1 | -16/+0 |
| | | |||||
| * | Merge branch 'bnewbold-match-scale' | Bryan Newbold | 2018-08-21 | 1 | -0/+2 |
| |\ | |||||
| | * | add a trap to ScoreJob | Bryan Newbold | 2018-08-20 | 1 | -0/+2 |
| | | | |||||
| * | | fix bugs/typos in HBaseColCountJob and HBaseStatusCountJob | Bryan Newbold | 2018-08-21 | 2 | -14/+7 |
| | | | |||||
| * | | distinction between status_code and status counting | Bryan Newbold | 2018-08-21 | 2 | -6/+75 |
| | | | |||||
| * | | add GrobidScorableDumpJob and basic test | Bryan Newbold | 2018-08-21 | 1 | -0/+124 |
| | | | |||||
| * | | Merge branch 'strings' | Bryan Newbold | 2018-08-21 | 2 | -0/+22 |
| |\ \ | | | | | | | | | | | | | | | | Resolved conflicts in: scalding/src/main/scala/sandcrawler/ScorableFeatures.scala | ||||
| | * | | Reads blacklist from file. | Ellen Spertus | 2018-08-20 | 2 | -0/+22 |
| | |/ | |||||
| * | | Created static factory method for ScorableCreations to deal with null. | Ellen Spertus | 2018-08-20 | 1 | -3/+3 |
| | | | |||||
| * | | Disabled scalastyle null checking where we want to test null values. | Ellen Spertus | 2018-08-20 | 1 | -0/+2 |
| | | | |||||
| * | | Reduced boilerplate code. | Ellen Spertus | 2018-08-20 | 1 | -11/+11 |
| |/ | |||||
| * | change slugification behavior to not split on colon | Bryan Newbold | 2018-08-15 | 2 | -23/+23 |
| | | |||||
| * | add a stub title blacklist | Bryan Newbold | 2018-08-15 | 1 | -0/+6 |
| | | |||||
| * | handle null status_code lines | Bryan Newbold | 2018-08-15 | 1 | -3/+7 |
| | | |||||
| * | unrelated TODO about testing with null HBase values | Bryan Newbold | 2018-08-15 | 1 | -0/+1 |
| | | |||||
| * | scorable: test for more punctuation removal | Bryan Newbold | 2018-08-15 | 1 | -0/+8 |
| | | |||||
| * | crossref: test for empty-string title | Bryan Newbold | 2018-08-15 | 1 | -0/+6 |
| | | |||||
| * | scorable: test for null strings | Bryan Newbold | 2018-08-15 | 1 | -0/+5 |
| | | |||||
| * | grobid scoring: status_code as signed int, not string | Bryan Newbold | 2018-08-15 | 1 | -2/+3 |
| | | |||||
| * | Fixed style problems (or disabled warning when appropriate) for tests. | Ellen Spertus | 2018-08-14 | 9 | -100/+128 |
| | | |||||
| * | Minor improvements. | Ellen Spertus | 2018-08-14 | 1 | -10/+7 |
| | | |||||
| * | Now ignores grobid entries with status other than 200. | Ellen Spertus | 2018-08-14 | 2 | -17/+32 |
| | | |||||
| * | Factored out ScorableFeatures. | Ellen Spertus | 2018-08-13 | 2 | -32/+37 |
| | | |||||
| * | Pipeline works, all tests pass, no scalastyle errors. | Ellen Spertus | 2018-08-13 | 2 | -33/+53 |
| | | |||||
| * | Snapshot before changing Scorable to find bug. | Ellen Spertus | 2018-08-12 | 3 | -17/+23 |
| | | |||||
| * | Added back file I shouldn't have deleted. | Ellen Spertus | 2018-08-12 | 1 | -0/+89 |
| | | |||||
| * | Tests pass. | Ellen Spertus | 2018-08-12 | 3 | -104/+33 |
| | | |||||
| * | It compiles. | Ellen Spertus | 2018-08-11 | 2 | -22/+23 |
| | | |||||
| * | Tests pass. Still have changes to do but made huge progress. | Ellen Spertus | 2018-08-10 | 2 | -2/+3 |
| | | |||||
 |
index : sandcrawler | |
| [no description] |
| aboutsummaryrefslogtreecommitdiffstats |