index
:
sandcrawler
bnewbold-args
bnewbold-backfill
bnewbold-persist-grobid-errors
bnewbold-refactor-loggging
master
trawler
[no description]
about
summary
refs
log
tree
commit
diff
stats
log msg
author
committer
range
Commit message (
Expand
)
Author
Age
Files
Lines
...
*
improve ingest worker remote failure behavior
Bryan Newbold
2019-11-13
1
-5
/
+12
*
have SPN client differentiate between SPN and remote errors
Bryan Newbold
2019-11-13
2
-3
/
+11
*
correct ingest-file consumer group
Bryan Newbold
2019-11-13
1
-1
/
+1
*
new kafka topics for file ingest
Bryan Newbold
2019-11-13
1
-0
/
+15
*
ingest/backfill notes
Bryan Newbold
2019-11-13
3
-0
/
+47
*
notes about running 'regrobid' batches manually (not kafka)
Bryan Newbold
2019-11-13
1
-0
/
+41
*
commit old notes about munging GROBID output
Bryan Newbold
2019-11-13
1
-0
/
+70
*
add structure of ingest proposal
Bryan Newbold
2019-11-13
1
-0
/
+129
*
add basic sandcrawler worker (kafka)
Bryan Newbold
2019-11-13
1
-0
/
+74
*
note that kafka_grobid.py is deprecated
Bryan Newbold
2019-11-13
1
-0
/
+3
*
rename FileIngestWorker
Bryan Newbold
2019-11-13
3
-10
/
+16
*
refactor consume_topic name out of make_kafka_consumer()
Bryan Newbold
2019-11-13
1
-5
/
+5
*
more progress on file ingest
Bryan Newbold
2019-11-13
4
-17
/
+75
*
add note to CDX backfill script that we should be filtering (oops)
Bryan Newbold
2019-11-12
1
-0
/
+1
*
SQL stats and commands (mostly from sept 2019)
Bryan Newbold
2019-11-12
4
-0
/
+96
*
much progress on file ingest path
Bryan Newbold
2019-10-22
6
-335
/
+338
*
remove spurious debug print from grobid2json
Bryan Newbold
2019-10-22
1
-1
/
+1
*
new/additional GWB CDX filter scripts
Bryan Newbold
2019-10-17
7
-0
/
+142
*
we do actually want consolidateHeader=2, not 1
Bryan Newbold
2019-10-04
2
-4
/
+4
*
remove any trailing newline
Bryan Newbold
2019-10-04
1
-2
/
+2
*
grobid: consolidateHeaders typo
Bryan Newbold
2019-10-04
1
-1
/
+1
*
grobid_tool: don't wrap multiprocess if we don't need to
Bryan Newbold
2019-10-04
1
-2
/
+4
*
disable citation consolidation by default
Bryan Newbold
2019-10-04
1
-1
/
+1
*
grobid-output-pg, not grobid-output-json
Bryan Newbold
2019-10-04
1
-4
/
+2
*
grobid_tool: don't always insert multi wrapper
Bryan Newbold
2019-10-04
1
-6
/
+13
*
grobid2json: language_code
Bryan Newbold
2019-10-04
2
-1
/
+7
*
fix GROBID POST flags
Bryan Newbold
2019-10-04
1
-1
/
+3
*
workers: better generic batch-size arg handling
Bryan Newbold
2019-10-03
1
-0
/
+6
*
handle GROBID fetch empty blob condition
Bryan Newbold
2019-10-03
1
-1
/
+2
*
update kafka topic listings
Bryan Newbold
2019-10-03
1
-21
/
+35
*
grobid_affiliations fix from prod, and usage example
Bryan Newbold
2019-10-02
1
-0
/
+5
*
deliver_dumpgrobid_to_s3: typo fix from old prod
Bryan Newbold
2019-10-02
1
-3
/
+4
*
grobid affiliation extractor (script)
Bryan Newbold
2019-10-02
1
-0
/
+47
*
python tests for pusher classes
Bryan Newbold
2019-10-02
2
-0
/
+28
*
have grobidworker error status indicate issues instead of bailing
Bryan Newbold
2019-10-02
1
-4
/
+13
*
grobid_tool.py example usage in docstring
Bryan Newbold
2019-10-02
1
-0
/
+6
*
add tests for affiliation extraction
Bryan Newbold
2019-10-02
2
-1
/
+25
*
have grobid2json extract full names and affiliations
Bryan Newbold
2019-10-02
1
-5
/
+27
*
more counts and bugfixes in grobid_tool
Bryan Newbold
2019-09-26
3
-5
/
+7
*
off-by-one error in batch sizes
Bryan Newbold
2019-09-26
1
-1
/
+1
*
small improvements to GROBID tool
Bryan Newbold
2019-09-26
2
-2
/
+10
*
lots of grobid tool implementation (still WIP)
Bryan Newbold
2019-09-26
8
-16
/
+748
*
gitlab CI: run both python and python_hadoop tests
Bryan Newbold
2019-09-25
1
-1
/
+6
*
pylint as part of pytest; update lint config
Bryan Newbold
2019-09-25
2
-1
/
+17
*
pipfile update
Bryan Newbold
2019-09-25
2
-241
/
+70
*
test of GROBID client
Bryan Newbold
2019-09-25
1
-0
/
+53
*
move a bunch of random old scripts to subdir
Bryan Newbold
2019-09-25
9
-0
/
+0
*
get rid of old xml2json
Bryan Newbold
2019-09-25
1
-7
/
+0
*
update README with new folders
Bryan Newbold
2019-09-25
1
-4
/
+10
*
point 'please' to python_hadoop
Bryan Newbold
2019-09-25
1
-4
/
+4
[prev]
[next]