sandcrawler - [no description]

	Commit message (Collapse)	Author	Age	Files	Lines
...
*	commented special modes for dump_unextracted_pdf.sql	Bryan Newbold	2020-06-25	1	-1/+4
\|
*	pdfextract support in ingest worker	Bryan Newbold	2020-06-25	3	-1/+66
\|
*	poppler: correct RGBA buffer endian-ness	Bryan Newbold	2020-06-25	2	-2/+2
\|
*	2020-05_pubmed ingest notes (short)	Bryan Newbold	2020-06-25	1	-0/+10
\|
*	commit old notes on a one-off CDX table cleanup	Bryan Newbold	2020-06-25	1	-0/+34
\|
*	commit old (2020-02) pdftrio commands	Bryan Newbold	2020-06-25	1	-0/+162
\|
*	pdftrio SQL queries	Bryan Newbold	2020-06-25	1	-0/+65
\|
*	SQL commands for re-trying PDF ingests	Bryan Newbold	2020-06-25	1	-0/+158
\|
*	start of RUNBOOK commands	Bryan Newbold	2020-06-25	1	-0/+44
\|
*	unextracted PDF job dump command	Bryan Newbold	2020-06-25	1	-0/+16
\|
*	pdfextract_tool fixes from prod usage	Bryan Newbold	2020-06-25	2	-3/+6
\|
*	fix tests for page0_height/width	Bryan Newbold	2020-06-25	1	-2/+2
\|
*	pdfextract: fix pdf_extra key names	Bryan Newbold	2020-06-25	1	-2/+2
\|
*	ensure pdf_meta isn't passed an empty dict()	Bryan Newbold	2020-06-25	1	-1/+4
\|
*	args.kafka_env refactor didn't happen (yet)	Bryan Newbold	2020-06-25	1	-2/+2
\|
*	s3-only mode persist workers use different consumer group	Bryan Newbold	2020-06-25	1	-2/+8
\|
*	changes from prod	Bryan Newbold	2020-06-25	2	-4/+18
\|
*	PDF extraction kafka topics	Bryan Newbold	2020-06-25	1	-0/+24
\|
*	sandcrawler_worker: remove duplicate run_pdf_extract()	Bryan Newbold	2020-06-25	1	-29/+0
\|
*	gitlab-ci: install most recent cmake via PPA	Bryan Newbold	2020-06-25	1	-3/+7
\|
*	pdfextract worker	Bryan Newbold	2020-06-25	1	-1/+34
\|
*	pdfextract: don't compress thumbnail output	Bryan Newbold	2020-06-25	1	-1/+1
\|
*	pipenv: python-poppler 0.2.1	Bryan Newbold	2020-06-25	2	-49/+51
\|
*	add xenial poppler PPA (PDF parsing)	Bryan Newbold	2020-06-18	1	-0/+1
\|
*	fixes and tweaks from testing locally	Bryan Newbold	2020-06-17	6	-18/+134
\|
*	fixes to pdfextract_tool	Bryan Newbold	2020-06-17	1	-12/+8
\|
*	tweak pdf_meta SQL schema	Bryan Newbold	2020-06-17	2	-5/+31
\|
*	add a docker-compose file for local development	Bryan Newbold	2020-06-17	2	-0/+50
\|
*	write up blob storage (w/ seaweedfs)	Bryan Newbold	2020-06-17	2	-0/+95
\| \| \| \|	This follows-on to Martin's earlier proposal.
*	tweak kafka topic names and seaweedfs layout	Bryan Newbold	2020-06-17	3	-13/+16
\|
*	make process_pdf() more robust to parse errors	Bryan Newbold	2020-06-17	1	-5/+29
\|
*	note about text layout with pdf extraction	Bryan Newbold	2020-06-17	1	-0/+8
\|
*	lint fixes	Bryan Newbold	2020-06-17	4	-7/+9
\|
*	rename pdf tools to pdfextract	Bryan Newbold	2020-06-17	3	-0/+0
\|
*	coverage-html makefile target	Bryan Newbold	2020-06-17	1	-0/+3
\|
*	add debian dependencies for python-poppler to CI setup	Bryan Newbold	2020-06-17	1	-1/+1
\|
*	pdf thumbnail+text+meta proposal	Bryan Newbold	2020-06-17	1	-0/+327
\|
*	ignore PIL deprecation warnings	Bryan Newbold	2020-06-17	1	-0/+1
\|
*	partial test coverage of pdf extract worker	Bryan Newbold	2020-06-17	2	-6/+70
\|
*	fix coverage command	Bryan Newbold	2020-06-17	2	-2/+4
\|
*	update grobid2json with type annotations	Bryan Newbold	2020-06-17	1	-94/+110
\|
*	remove unused common.py	Bryan Newbold	2020-06-17	2	-139/+0
\|
*	better DeprecationWarning filters	Bryan Newbold	2020-06-17	1	-3/+4
\|
*	update Makefile from fatcat-scholar tweaks/commands	Bryan Newbold	2020-06-17	1	-3/+21
\|
*	WIP on pdf_tool.py	Bryan Newbold	2020-06-17	1	-0/+137
\|
*	add new pdf workers/persisters	Bryan Newbold	2020-06-17	4	-2/+214
\|
*	pdf: mypy and typo fixes	Bryan Newbold	2020-06-17	2	-15/+22
\|
*	workers: refactor to pass key to process()	Bryan Newbold	2020-06-17	6	-20/+28
\|
*	pipenv: correct poppler; update lockfile	Bryan Newbold	2020-06-16	2	-76/+255
\|
*	pipenv: flake8, pytype, black	Bryan Newbold	2020-06-16	1	-0/+7
\|