sandcrawler/python/scripts, branch master
[no description]
https://git.bnewbold.net/sandcrawler/atom?h=master
2022-09-28T22:40:35+00:00
update oai-pmh ingest request transform script
2022-09-28T22:40:35+00:00
Bryan Newbold
bnewbold@archive.org
2022-09-28T22:40:35+00:00
urn:sha1:dfd4605d84712eccb95a63e50b0bcb343642b433
doaj and unpaywall transforms: more domains to skip
2022-07-21T01:02:56+00:00
Bryan Newbold
bnewbold@archive.org
2022-07-21T01:02:56+00:00
urn:sha1:a72019e6e788be64420719c5045e40614098c106
row2json script: fix argument type
2022-07-15T20:19:12+00:00
Bryan Newbold
bnewbold@archive.org
2022-07-15T20:19:12+00:00
urn:sha1:3f8fed325a3dd8d51652dffab89880c1cf25656b
row2json script: add flag to enable recrawling
2022-07-15T20:07:10+00:00
Bryan Newbold
bnewbold@archive.org
2022-07-15T20:07:10+00:00
urn:sha1:ce386a7a1f38c2c6289ce9d0bde269b4b1349154
more sentry config changes
2022-02-25T19:45:45+00:00
Bryan Newbold
bnewbold@archive.org
2022-02-25T19:45:45+00:00
urn:sha1:a0766e6a0100694434ddb2a4a895287806049ed8
switch from 'raven' to 'sentry-sdk'
2022-02-25T00:35:10+00:00
Bryan Newbold
bnewbold@archive.org
2022-02-25T00:35:10+00:00
urn:sha1:7f7846b99042897afd5916b9263320c0e2775706
add CDX sha1hex lookup/fetch helper script
2021-11-30T23:29:41+00:00
Bryan Newbold
bnewbold@archive.org
2021-11-30T23:29:41+00:00
urn:sha1:0328598e3b643edd0a2033ca97c607f596dfb092
remove grobid2json helper file, replace with grobid_tei_xml
2021-10-28T02:10:35+00:00
Bryan Newbold
bnewbold@archive.org
2021-10-28T02:10:35+00:00
urn:sha1:a0e275a4bad46ef41585f0207d6dfa1e3c38bc35
make fmt (black 21.9b0)
2021-10-28T01:50:17+00:00
Bryan Newbold
bnewbold@archive.org
2021-10-28T01:50:17+00:00
urn:sha1:826c7538e091fac14d987a3cd654975da964e240
make fmt
2021-10-26T19:54:37+00:00
Bryan Newbold
bnewbold@archive.org
2021-10-26T19:54:37+00:00
urn:sha1:05bd7cbcc62588e431c5efd533189e246b2a997e