aboutsummaryrefslogtreecommitdiffstats
path: root/python
Commit message (Collapse)AuthorAgeFilesLines
* pubmed: update docsMartin Czygan2021-07-171-2/+3
|
* pubmed: do not fail when accessing missing fileMartin Czygan2021-07-171-2/+8
| | | | | | | after a sync gap (e.g. 06/07 2021) harvester wanted to fetch a file, that was not on the server (any more) - do not fail in this case we'll need to backfill missing records via full data dump
* pubmed: reconnect on errorMartin Czygan2021-07-161-4/+30
| | | | | | | | | ftp retrieval would run but fail with EOFError on /pubmed/updatefiles/pubmed21n1328_stats.html - not able to find the root cause; using a fresh client, the exact same file would work just fine. So when we retry, we reconnect on failure. Refs: sentry #91102.
* web: fix flask/werkzeug encoding for mediawiki oauthBryan Newbold2021-07-131-1/+4
|
* web: fix missing ext_ids default for deleted entity viewBryan Newbold2021-07-131-1/+1
|
* web: fix 'file' entity edit form linksBryan Newbold2021-07-021-1/+1
|
* web: missing trailing parensBryan Newbold2021-07-021-1/+1
|
* web: PMCID external link improvementBryan Newbold2021-07-022-2/+2
|
* Merge branch 'bnewbold-more-doi-lower' into 'master'Martin Czygan2021-07-023-3/+8
|\ | | | | | | | | more consistent and defensive lower-casing of DOIs See merge request webgroup/fatcat!109
| * more consistent and defensive lower-casing of DOIsBryan Newbold2021-06-233-3/+8
| | | | | | | | | | | | | | After noticing more upper/lower ambiguity in production. In particular, we have some old ingest requests in sandcrawler DB, which get re-submitted/re-tried, which have capitalized DOIs in the link source id field.
* | tests: small citeproc style changes (to match Pipfile.lock update)Bryan Newbold2021-06-232-3/+4
| |
* | pipenv: regenerate lock fileBryan Newbold2021-06-231-26/+68
| |
* | pipenv: add pydantic; add surt; narrow dynaconfBryan Newbold2021-06-231-1/+3
|/
* datacite: more careful title string access; fixes sentry #88350Martin Czygan2021-06-114-2/+97
| | | | | Caused by a partial "title entry without title" coming *first* (e.g. just holding, e.g. a language, like: {'lang': 'da'}
* clean_doi() should lower-case returned DOIBryan Newbold2021-06-071-1/+4
| | | | | | | | | | Code in a number of places (including Pubmed importer) assumed that this was already lower-casing DOIs, resulting in some broken metadata getting created. See also: https://github.com/internetarchive/fatcat/issues/83 This is just the first step of mitigation.
* web: fix DOAJ article links (remove trailing slash)Bryan Newbold2021-06-041-1/+1
|
* dblp tests: skip redundant seek(0)Bryan Newbold2021-06-031-6/+1
|
* ingest: swap ingest and file checks, to result in clearer stats/counts of ↵Bryan Newbold2021-06-031-2/+2
| | | | skipping
* ingest: don't accept mag and s2 URLsBryan Newbold2021-06-031-4/+4
|
* bump fuzzycat dependency to 0.1.21Bryan Newbold2021-06-022-20/+18
|
* web: fix spacing for doaj/dblp identifiers in SERPBryan Newbold2021-05-311-1/+1
|
* ingest: don't 'track_total_hits' for ES 7.x count()Bryan Newbold2021-05-311-1/+1
|
* web: bugfix dblp vs. doaj display logicBryan Newbold2021-05-311-1/+1
|
* update fuzzycat to 0.1.20Bryan Newbold2021-05-312-31/+94
|
* makefile: add pylint -E invocation to 'make lint', to match CIBryan Newbold2021-05-251-0/+1
|
* skip pylint on 'assigning-non-slot' warnings in Flask 2.0Bryan Newbold2021-05-251-2/+2
| | | | | | | | | | | | | | The 'permanent' field is still valid to set to a boolean in Flask 2.0; not sure why pylint is unhappy in CI (causing test failures). Don't see any problem running test suite locally. Flask API docs: https://flask.palletsprojects.com/en/2.0.x/api/?highlight=permanent#flask.session.permanent And code (recent master branch): https://github.com/pallets/flask/blob/4240ace59710d86c478111affd4ad6fb4c8cad9e/src/flask/sessions.py#L20
* changelog worker: fix file/fileset typo, caught by lintBryan Newbold2021-05-251-1/+1
| | | | | This would have been resulting in some releases not getting re-indexed into search.
* small python lint fixes (no behavior change)Bryan Newbold2021-05-255-6/+4
|
* bump Flask to 2.x; other deps upgraded automaticallyBryan Newbold2021-05-212-152/+167
|
* ingest: add per-container ingest type overridesBryan Newbold2021-05-212-1/+23
|
* fix arabesque sqlite3 examples to have 14-digit timestampsBryan Newbold2021-05-211-0/+0
|
* arabesque importer: ensure full 14-digit timestampsBryan Newbold2021-05-211-1/+3
|
* Andrew W. Mellon FoundationBryan Newbold2021-05-182-3/+3
|
* Merge branch 'bnewbold-pipenv-cleanup' into 'master'bnewbold2021-04-232-327/+277
|\ | | | | | | | | pipenv cleanup See merge request webgroup/fatcat!104
| * pipenv: re-lock projectBryan Newbold2021-04-191-301/+253
| |
| * pipenv: constrain most package versions to at least majorBryan Newbold2021-04-191-24/+24
| | | | | | | | | | | | | | | | | | Don't have a complete policy with this change, just locking things down a bit more so small package additions and updates don't end up upgrading some small dependency to a major new backwards-incompatible version. Also, correct bs4 -> beautifulsoup4 (bs4 is the import name, not the package name)
| * pipenv: remove unused pg-view and pykafka librariesBryan Newbold2021-04-191-2/+0
| |
* | web: fix edit form style guide linksBryan Newbold2021-04-202-4/+4
|/
* transforms: fix 'display_ame' typoBryan Newbold2021-04-191-2/+2
|
* web: expand release creators in more situationsBryan Newbold2021-04-192-2/+2
|
* fix public API linksBryan Newbold2021-04-151-2/+2
|
* fix 'colected' typosBryan Newbold2021-04-131-1/+1
| | | | Thanks for the catch martin
* prefer contrib.creator.display_name over contrib.raw_nameBryan Newbold2021-04-124-9/+17
| | | | | | | | These will be getting updates from ORCID and are usually more complete and more correct for display, attribution, and search purposes. Might need to tweak fuzzycat code to handle multiple names at the verification stage.
* make dblp tests more robustBryan Newbold2021-04-121-2/+11
| | | | | | These were causing a lot of spurious errors in local development. Not sure these tweaks will entirely fix the problem.
* web: show file size not known, when it isn'tBryan Newbold2021-04-121-0/+2
| | | | This is mostly to prevent showing an empty metadata box
* web: better logic for showing 'save-paper-now' linkBryan Newbold2021-04-121-0/+2
|
* web: include DOI in share-your-paper URL, when possibleBryan Newbold2021-04-121-2/+8
|
* web: consistent public API URLsBryan Newbold2021-04-126-14/+9
|
* web: improve preservation holdings display for containersBryan Newbold2021-04-121-10/+22
|
* web: improve access button HTMLBryan Newbold2021-04-122-3/+2
|
generated by cgit v1.2.3 (git 2.39.1) at 2024-11-12 02:08:53 +0000