index
:
sandcrawler
bnewbold-args
bnewbold-backfill
bnewbold-persist-grobid-errors
bnewbold-refactor-loggging
master
trawler
[no description]
about
summary
refs
log
tree
commit
diff
stats
log msg
author
committer
range
Commit message (
Expand
)
Author
Age
Files
Lines
*
yet another 'bad' PDF sha1hex
Bryan Newbold
2020-07-27
1
-0
/
+1
*
use new SPNv2 'skip_first_archive' param
Bryan Newbold
2020-07-22
1
-0
/
+1
*
MAG 2020-07 ingest notes
Bryan Newbold
2020-07-08
1
-0
/
+159
*
add more slow PDF hashes
Bryan Newbold
2020-07-05
1
-0
/
+2
*
add another bad PDF sha1hex
Bryan Newbold
2020-07-02
1
-0
/
+1
*
seaweedfs proposal: fix typos and wording
Martin Czygan
2020-07-01
1
-9
/
+11
*
another bad PDF SHA-1
Bryan Newbold
2020-06-30
1
-0
/
+1
*
hack to unblock thumbnail processing pipeline
Bryan Newbold
2020-06-29
1
-0
/
+16
*
customize timeout per worker; 120sec for pdf-extract
Bryan Newbold
2020-06-29
3
-2
/
+4
*
handle empty fetched blob
Bryan Newbold
2020-06-27
1
-1
/
+6
*
CDX KeyError as WaybackError from fetch worker
Bryan Newbold
2020-06-26
1
-1
/
+1
*
handle None 'metadata' field correctly
Bryan Newbold
2020-06-26
1
-1
/
+1
*
handle non-success case of parsing extract from JSON/dict
Bryan Newbold
2020-06-26
1
-1
/
+1
*
report revisit non-200 as a WaybackError
Bryan Newbold
2020-06-26
1
-7
/
+7
*
Revert "simpler handling of null PDF text pages"
Bryan Newbold
2020-06-25
1
-4
/
+11
*
simpler handling of null PDF text pages
Bryan Newbold
2020-06-25
1
-11
/
+4
*
pdfextract: attributerror with text extraction
Bryan Newbold
2020-06-25
1
-4
/
+12
*
catch UnicodeDecodeError in pdfextract
Bryan Newbold
2020-06-25
1
-1
/
+10
*
don't nest generic fetch errors under pdf_trio
Bryan Newbold
2020-06-25
1
-12
/
+6
*
pdfextract: handle too-large fulltext
Bryan Newbold
2020-06-25
1
-0
/
+17
*
another bad/non PDF test; catch correct error
Bryan Newbold
2020-06-25
2
-1
/
+6
*
pdfextract: catch poppler.LockedDocumentError
Bryan Newbold
2020-06-25
1
-1
/
+1
*
commented special modes for dump_unextracted_pdf.sql
Bryan Newbold
2020-06-25
1
-1
/
+4
*
pdfextract support in ingest worker
Bryan Newbold
2020-06-25
3
-1
/
+66
*
poppler: correct RGBA buffer endian-ness
Bryan Newbold
2020-06-25
2
-2
/
+2
*
2020-05_pubmed ingest notes (short)
Bryan Newbold
2020-06-25
1
-0
/
+10
*
commit old notes on a one-off CDX table cleanup
Bryan Newbold
2020-06-25
1
-0
/
+34
*
commit old (2020-02) pdftrio commands
Bryan Newbold
2020-06-25
1
-0
/
+162
*
pdftrio SQL queries
Bryan Newbold
2020-06-25
1
-0
/
+65
*
SQL commands for re-trying PDF ingests
Bryan Newbold
2020-06-25
1
-0
/
+158
*
start of RUNBOOK commands
Bryan Newbold
2020-06-25
1
-0
/
+44
*
unextracted PDF job dump command
Bryan Newbold
2020-06-25
1
-0
/
+16
*
pdfextract_tool fixes from prod usage
Bryan Newbold
2020-06-25
2
-3
/
+6
*
fix tests for page0_height/width
Bryan Newbold
2020-06-25
1
-2
/
+2
*
pdfextract: fix pdf_extra key names
Bryan Newbold
2020-06-25
1
-2
/
+2
*
ensure pdf_meta isn't passed an empty dict()
Bryan Newbold
2020-06-25
1
-1
/
+4
*
args.kafka_env refactor didn't happen (yet)
Bryan Newbold
2020-06-25
1
-2
/
+2
*
s3-only mode persist workers use different consumer group
Bryan Newbold
2020-06-25
1
-2
/
+8
*
changes from prod
Bryan Newbold
2020-06-25
2
-4
/
+18
*
PDF extraction kafka topics
Bryan Newbold
2020-06-25
1
-0
/
+24
*
sandcrawler_worker: remove duplicate run_pdf_extract()
Bryan Newbold
2020-06-25
1
-29
/
+0
*
gitlab-ci: install most recent cmake via PPA
Bryan Newbold
2020-06-25
1
-3
/
+7
*
pdfextract worker
Bryan Newbold
2020-06-25
1
-1
/
+34
*
pdfextract: don't compress thumbnail output
Bryan Newbold
2020-06-25
1
-1
/
+1
*
pipenv: python-poppler 0.2.1
Bryan Newbold
2020-06-25
2
-49
/
+51
*
add xenial poppler PPA (PDF parsing)
Bryan Newbold
2020-06-18
1
-0
/
+1
*
fixes and tweaks from testing locally
Bryan Newbold
2020-06-17
6
-18
/
+134
*
fixes to pdfextract_tool
Bryan Newbold
2020-06-17
1
-12
/
+8
*
tweak pdf_meta SQL schema
Bryan Newbold
2020-06-17
2
-5
/
+31
*
add a docker-compose file for local development
Bryan Newbold
2020-06-17
2
-0
/
+50
[next]