sandcrawler - [no description]

	Commit message (Collapse)	Author	Age	Files	Lines
...
*	support forwarding url types other than pdf_url	Bryan Newbold	2020-01-09	1	-4/+5
\|
*	refactor ingest to a loop, allowing multiple hops	Bryan Newbold	2020-01-09	1	-25/+48
\|
*	lots of progress on wayback refactoring	Bryan Newbold	2020-01-09	1	-11/+15
\| \| \| \| \| \|	- too much to list - canonical flags to control crawling - cdx_to_dict helper
*	wrap up basic (locally testable) ingest refactor	Bryan Newbold	2020-01-09	1	-159/+196
\|
*	remove SPNv1 code paths	Bryan Newbold	2020-01-07	1	-30/+24
\|
*	refactor: use print(..., file=sys.stderr)	Bryan Newbold	2019-12-18	1	-4/+4
\| \| \| \|	Should use logging soon, but this seems more idiomatic in the meanwhile.
*	handle wayback fetch redirect loop in ingest code	Bryan Newbold	2019-11-14	1	-2/+5
\|
*	handle WaybackError during ingest	Bryan Newbold	2019-11-14	1	-0/+4
\|
*	start of hrmars.com ingest support	Bryan Newbold	2019-11-14	1	-2/+5
\|
*	treat failure to get terminal capture as a SavePageNowError	Bryan Newbold	2019-11-13	1	-1/+1
\|
*	handle wayback client return status correctly	Bryan Newbold	2019-11-13	1	-2/+2
\|
*	allow way more errors in SPN path	Bryan Newbold	2019-11-13	1	-2/+11
\|
*	fix lint errors	Bryan Newbold	2019-11-13	1	-1/+1
\|
*	improve ingest worker remote failure behavior	Bryan Newbold	2019-11-13	1	-5/+12
\|
*	rename FileIngestWorker	Bryan Newbold	2019-11-13	1	-5/+9
\|
*	more progress on file ingest	Bryan Newbold	2019-11-13	1	-10/+37
\|
*	much progress on file ingest path	Bryan Newbold	2019-10-22	1	-0/+150