index
:
sandcrawler
bnewbold-args
bnewbold-backfill
bnewbold-persist-grobid-errors
bnewbold-refactor-loggging
master
trawler
[no description]
about
summary
refs
log
tree
commit
diff
stats
log msg
author
committer
range
path:
root
/
scalding
Commit message (
Expand
)
Author
Age
Files
Lines
*
Added accent removal to titleToSlug().
Ellen Spertus
2018-07-28
2
-2
/
+51
*
Added tests (both pass) to try to understand crash when run on real datwa.
Ellen Spertus
2018-07-28
1
-0
/
+9
*
Commented out debug() calls.
Ellen Spertus
2018-07-26
1
-2
/
+1
*
Made changes suggested in MR.
Ellen Spertus
2018-07-26
1
-30
/
+41
*
Computes and outputs (score, sha1, doi, grobidTitle, crossrefTitle).
Ellen Spertus
2018-07-26
2
-33
/
+71
*
Made progress on crossrefPipe.
Ellen Spertus
2018-07-25
2
-12
/
+19
*
Fixed bug with reading from TextLine. (Thanks, Bryan\!) Still had to comment ...
Ellen Spertus
2018-07-25
2
-11
/
+10
*
Show full stack traces.
Ellen Spertus
2018-07-25
1
-0
/
+1
*
Restored my old tests. Commented out broken tests.
Ellen Spertus
2018-07-25
2
-43
/
+68
*
Checked all fields of first entry in grobid pipe.
Ellen Spertus
2018-07-25
1
-2
/
+4
*
Grobid entries without legal slugs are removed from the pipe.
Ellen Spertus
2018-07-25
2
-10
/
+22
*
Figured out string conversion. Tests pass. Still WIP.
Ellen Spertus
2018-07-25
2
-17
/
+21
*
WIP. I'm having problems converting between ImmutableBytesWritable and String.
Ellen Spertus
2018-07-24
2
-23
/
+84
*
Clean-up
Ellen Spertus
2018-07-24
2
-19
/
+28
*
Added grobidToSlug().
Ellen Spertus
2018-07-24
2
-9
/
+91
*
Changed return type of grobidToSlug() to Option[String].
Ellen Spertus
2018-07-24
2
-10
/
+20
*
grobidToSlug() seems to work, including parsing of valid JSON strings.
Ellen Spertus
2018-07-24
2
-0
/
+131
*
Undid changes that did not belong in this branch.
Ellen Spertus
2018-07-19
3
-304
/
+0
*
Merge branch 'scalastyle' of git.archive.org:webgroup/sandcrawler into scalas...
Ellen Spertus
2018-07-19
2
-6
/
+9
|
\
|
*
Cleanup after merge master. Note: I am not fixing warnings in CdxBackfillJob....
Ellen Spertus
2018-07-19
8
-30
/
+341
|
|
\
|
|
*
Merge branch 'bnewbold-backfill' of git.archive.org:webgroup/sandcrawler
Ellen Spertus
2018-07-19
5
-11
/
+322
|
|
|
\
|
|
|
*
more debugging notes
Bryan Newbold
2018-07-17
1
-0
/
+14
|
|
|
*
refactor CdxBackfillJob to be all Typed (failed tests)
Bryan Newbold
2018-07-17
1
-21
/
+46
|
|
|
*
more complete CdxBackfillJob (untested)
Bryan Newbold
2018-07-17
1
-2
/
+68
|
|
|
*
fix confusing whitespace in HBaseStatusCountTest
Bryan Newbold
2018-07-17
1
-11
/
+11
|
|
|
*
add buildSink() method for writing to HBase
Bryan Newbold
2018-07-17
1
-0
/
+7
|
|
|
*
early work on scalding CDX backfill
Bryan Newbold
2018-07-17
2
-0
/
+199
|
*
|
|
Fixed mistake found while testing.
Ellen Spertus
2018-07-19
1
-1
/
+1
|
*
|
|
Improved style and style checking.
Ellen Spertus
2018-07-19
6
-25
/
+54
*
|
|
|
Removed remaining warning.
Ellen Spertus
2018-07-19
1
-1
/
+1
*
|
|
|
Improved style and style checking.
Ellen Spertus
2018-07-19
5
-22
/
+39
*
|
|
|
more debugging notes
Bryan Newbold
2018-07-19
1
-0
/
+14
*
|
|
|
refactor CdxBackfillJob to be all Typed (failed tests)
Bryan Newbold
2018-07-19
1
-21
/
+46
*
|
|
|
more complete CdxBackfillJob (untested)
Bryan Newbold
2018-07-19
1
-2
/
+68
*
|
|
|
fix confusing whitespace in HBaseStatusCountTest
Bryan Newbold
2018-07-19
1
-11
/
+11
*
|
|
|
add buildSink() method for writing to HBase
Bryan Newbold
2018-07-19
1
-0
/
+7
*
|
|
|
early work on scalding CDX backfill
Bryan Newbold
2018-07-19
2
-0
/
+199
|
|
/
/
|
/
|
|
*
|
|
Added rule to warn about block imports. Removed block imports.
Ellen Spertus
2018-07-19
3
-3
/
+6
*
|
|
Eliminated example directories
Ellen Spertus
2018-07-17
1
-0
/
+8
*
|
|
Added import order checking. Fixed misordered imports in our code.
Ellen Spertus
2018-07-17
4
-4
/
+12
|
|
/
|
/
|
*
|
Merge branch 'scalastyle' into 'master'
bnewbold
2018-07-17
5
-4
/
+122
|
\
|
|
*
Turned off header checking.
Ellen Spertus
2018-07-17
1
-2
/
+2
|
*
Fixed style errors.
Ellen Spertus
2018-07-17
3
-4
/
+4
|
*
Installed scalastyle and created configuration file. Run with: sbt scalastyle
Ellen Spertus
2018-07-17
2
-0
/
+118
*
|
lint/cleanup fixes from review
Bryan Newbold
2018-07-17
1
-17
/
+12
|
/
*
refactor HBaseStatusCountJob to convert Long column
Bryan Newbold
2018-07-17
2
-18
/
+48
*
Merge branch 'bnewbold-args'
Bryan Newbold
2018-07-13
5
-16
/
+30
|
\
|
*
snake_case to lowerCamelCase
Bryan Newbold
2018-07-13
5
-18
/
+18
|
*
hbase table and zk host as arguments
Bryan Newbold
2018-06-17
5
-16
/
+30
*
|
Merge branch 'spertus-issue1' into 'master'
bnewbold
2018-07-13
2
-2
/
+2
|
\
\
|
|
/
|
/
|
[next]