Commit message (Expand) | Author | Age | Files | Lines | ||
---|---|---|---|---|---|---|
... | ||||||
* | another day, another bad PDF sha1 | Bryan Newbold | 2020-10-13 | 1 | -0/+1 | |
* | store no-capture URLs in terminal_url | Bryan Newbold | 2020-10-12 | 2 | -3/+3 | |
* | another bad PDF sha1 | Bryan Newbold | 2020-10-11 | 1 | -0/+1 | |
* | yet more bad sha1 PDFs to skip | Bryan Newbold | 2020-10-10 | 1 | -0/+20 | |
* | ingest: small bugfix to print pdfextract status on SUCCESS | Bryan Newbold | 2020-09-17 | 1 | -1/+1 | |
* | more bad PDF sha1 | Bryan Newbold | 2020-09-17 | 1 | -0/+2 | |
* | yet another broken PDF (sha1) | Bryan Newbold | 2020-09-16 | 1 | -0/+1 | |
* | html: handle JMIR URL pattern | Bryan Newbold | 2020-09-15 | 1 | -0/+6 | |
* | skip citation_pdf_url if it is a link loop | Bryan Newbold | 2020-09-14 | 1 | -2/+8 | |
* | html parse: add another generic fulltext pattern | Bryan Newbold | 2020-09-14 | 1 | -1/+10 | |
* | ingest: treat text/xml as XHTML in pdf ingest | Bryan Newbold | 2020-09-14 | 1 | -1/+1 | |
* | more bad SHA1 PDF | Bryan Newbold | 2020-09-02 | 1 | -0/+2 | |
* | another bad PDF sha1 | Bryan Newbold | 2020-09-01 | 1 | -0/+1 | |
* | another bad PDF sha1 | Bryan Newbold | 2020-08-24 | 1 | -0/+1 | |
* | html: handle embed with mangled 'src' attribute | Bryan Newbold | 2020-08-24 | 1 | -1/+1 | |
* | another bad PDF sha1 | Bryan Newbold | 2020-08-17 | 1 | -0/+1 | |
* | another bad PDF sha1 | Bryan Newbold | 2020-08-15 | 1 | -0/+1 | |
* | more bad sha1 | Bryan Newbold | 2020-08-14 | 1 | -0/+1 | |
* | yet more bad PDF sha1 | Bryan Newbold | 2020-08-14 | 1 | -0/+2 | |
* | more bad SHA1 | Bryan Newbold | 2020-08-13 | 1 | -0/+2 | |
* | yet another PDF sha1 | Bryan Newbold | 2020-08-12 | 1 | -0/+1 | |
* | another bad sha1; maybe the last for this batch? | Bryan Newbold | 2020-08-12 | 1 | -0/+1 | |
* | more bad sha1 | Bryan Newbold | 2020-08-11 | 1 | -0/+2 | |
* | additional loginwall patterns | Bryan Newbold | 2020-08-11 | 1 | -0/+2 | |
* | more SHA1 | Bryan Newbold | 2020-08-11 | 1 | -0/+2 | |
* | Revert "ingest: reduce CDX retry_sleep to 3.0 sec (after SPN)" | Bryan Newbold | 2020-08-11 | 1 | -1/+1 | |
* | ingest: reduce CDX retry_sleep to 3.0 sec (after SPN) | Bryan Newbold | 2020-08-11 | 1 | -1/+1 | |
* | ingest: actually use force_get flag with SPN | Bryan Newbold | 2020-08-11 | 1 | -0/+13 | |
* | check for simple URL patterns that are usually paywalls or loginwalls | Bryan Newbold | 2020-08-11 | 2 | -0/+29 | |
* | ingest: check for URL blocklist and cookie URL patterns on every hop | Bryan Newbold | 2020-08-11 | 1 | -0/+13 | |
* | refactor: force_get -> force_simple_get | Bryan Newbold | 2020-08-11 | 2 | -8/+8 | |
* | html: extract eprints PDF url (eg, ub.uni-heidelberg.de) | Bryan Newbold | 2020-08-11 | 1 | -0/+2 | |
* | extract PDF urls for e-periodica.ch | Bryan Newbold | 2020-08-10 | 1 | -0/+6 | |
* | more bad sha1 | Bryan Newbold | 2020-08-10 | 1 | -0/+2 | |
* | another bad PDF sha1 | Bryan Newbold | 2020-08-10 | 1 | -0/+1 | |
* | add hkvalidate.perfdrive.com to domain blocklist | Bryan Newbold | 2020-08-08 | 1 | -0/+3 | |
* | fix tests passing str as HTML | Bryan Newbold | 2020-08-08 | 1 | -3/+3 | |
* | add more HTML extraction tricks | Bryan Newbold | 2020-08-08 | 1 | -2/+29 | |
* | rwth-aachen.de HTML extract, and a generic URL guess method | Bryan Newbold | 2020-08-08 | 1 | -0/+15 | |
* | another PDF hash to skip | Bryan Newbold | 2020-08-08 | 1 | -0/+1 | |
* | another sha1 | Bryan Newbold | 2020-08-07 | 1 | -0/+1 | |
* | another sha1 | Bryan Newbold | 2020-08-06 | 1 | -0/+1 | |
* | and more bad sha1 | Bryan Newbold | 2020-08-06 | 1 | -0/+3 | |
* | more pdfextract skip sha1hex | Bryan Newbold | 2020-08-06 | 1 | -9/+12 | |
* | more bad PDF sha1; print sha1 before poppler extract | Bryan Newbold | 2020-08-05 | 1 | -0/+7 | |
* | spn2: skip js behavior (experiment) | Bryan Newbold | 2020-08-05 | 1 | -0/+1 | |
* | SPN2: ensure not fetching outlinks | Bryan Newbold | 2020-08-05 | 1 | -0/+1 | |
* | another bad PDF sha1 | Bryan Newbold | 2020-08-04 | 1 | -0/+1 | |
* | another PDF sha1hex | Bryan Newbold | 2020-07-27 | 1 | -0/+1 | |
* | yet another 'bad' PDF sha1hex | Bryan Newbold | 2020-07-27 | 1 | -0/+1 |