aboutsummaryrefslogtreecommitdiffstats
path: root/python/fatcat_tools/importers
Commit message (Expand)AuthorAgeFilesLines
* ingest importer: check that stage is consistent with releaseBryan Newbold2020-05-261-0/+5
* Merge remote-tracking branch 'github/master'Bryan Newbold2020-05-221-2/+2
|\
| * Indentity is not the same this as equality in PythonChristian Clauss2020-05-141-2/+2
* | importers: clarify handling of ApiExceptionBryan Newbold2020-05-223-4/+10
* | ingest importer: don't use glutton matchesBryan Newbold2020-05-221-3/+3
* | datacite: fix type errorMartin Czygan2020-04-221-1/+3
* | datacite: fix a raw name constraint violationMartin Czygan2020-04-201-0/+8
|/
* consistently use raw string prefix for regexBryan Newbold2020-04-171-1/+1
* pubmed: use untranslated title if translated not availableBryan Newbold2020-04-011-0/+6
* importers: replace newlines in get_text() stringsBryan Newbold2020-04-014-23/+25
* importers: more string/get_text swapsBryan Newbold2020-03-283-27/+27
* pubmed: bunch of .get_text() instead of .stringBryan Newbold2020-03-281-12/+12
* Merge pull request #53 from EdwardBetts/spellingbnewbold2020-03-273-7/+7
|\
| * Correct spelling mistakesEdward Betts2020-03-273-7/+7
* | datacite: nameIdentifier corner caseBryan Newbold2020-03-261-1/+2
* | jalc: avoid meaningless pages valuesBryan Newbold2020-03-231-4/+8
* | datacite: add year sanity restrictionsbnewbold2020-03-231-0/+7
* | pubmed: handle multiple ReferenceListBryan Newbold2020-03-201-1/+4
* | pubmed: update many more metadata fieldsBryan Newbold2020-03-191-0/+22
* | crossref: skip stub OUP titleBryan Newbold2020-03-191-0/+8
* | Merge branch 'martin-kafka-bs4-import' into 'master'Martin Czygan2020-03-102-1/+66
|\ \ | |/ |/|
| * common: use smaller batch size since XML parsing may be slowMartin Czygan2020-03-101-1/+1
| * pubmed ftp harvest and KafkaBs4XmlPusherMartin Czygan2020-02-192-1/+66
* | add some more domain/rel URL mappingsBryan Newbold2020-02-221-0/+9
* | Merge branch 'bnewbold-shadow-import'Bryan Newbold2020-02-192-0/+196
|\ \
| * | remove arabesque short wayback URL hackBryan Newbold2020-02-141-6/+0
| * | improve shadow import file url cleanup pathBryan Newbold2020-02-131-2/+12
| * | shadow import fixes from QA testingBryan Newbold2020-02-131-0/+6
| * | shadow import: more filtering of file_meta fieldsBryan Newbold2020-02-131-0/+10
| * | basic shadow importerBryan Newbold2020-02-132-0/+176
| |/
* | ingest import: fix edit_extra pathBryan Newbold2020-02-181-1/+1
* | ingest importer: edit_extra is a top-level keyBryan Newbold2020-02-181-1/+1
* | ingest import: allow short version of corpus namesBryan Newbold2020-02-181-0/+3
* | ingest importer: pass through link relBryan Newbold2020-02-181-1/+6
|/
* check ingest_request_source existance for SPN as well as ingestBryan Newbold2020-02-061-0/+3
* additional trusted link sourcesBryan Newbold2020-02-061-0/+3
* add mag and s2 as trusted link sourcesBryan Newbold2020-02-061-1/+1
* ingest worker: handle missing ingest_request_sourceBryan Newbold2020-02-061-0/+3
* datacite: prevent noneMartin Czygan2020-01-311-1/+1
* datacite: name shall not be NoneMartin Czygan2020-01-311-1/+1
* datacite: add exception for https://www.micropublication.org/Martin Czygan2020-01-311-0/+5
* datacite: do not skip records w/o dateMartin Czygan2020-01-311-2/+1
* datacite: improve docstringMartin Czygan2020-01-311-4/+4
* datacite: improve date handling and minor tweakMartin Czygan2020-01-301-19/+42
* fix KafkaError worker reporting for partition errorsBryan Newbold2020-01-291-1/+1
* fix trivial typo in file importerBryan Newbold2020-01-201-1/+1
* ingest: improve tests, support old ingest resultsBryan Newbold2020-01-151-3/+12
* update ingest worker for schema tweaksBryan Newbold2020-01-151-8/+15
* ingest: allow more sources to auto-importBryan Newbold2020-01-151-1/+2
* datacite: skip records without a doiMartin Czygan2020-01-131-0/+4
generated by cgit v1.2.3 (git 2.39.1) at 2024-11-24 11:18:37 +0000