summaryrefslogtreecommitdiffstats
path: root/python
Commit message (Collapse)AuthorAgeFilesLines
* datacite: more careful title string access; fixes sentry #88350Martin Czygan2021-06-114-2/+97
| | | | | Caused by a partial "title entry without title" coming *first* (e.g. just holding, e.g. a language, like: {'lang': 'da'}
* clean_doi() should lower-case returned DOIBryan Newbold2021-06-071-1/+4
| | | | | | | | | | Code in a number of places (including Pubmed importer) assumed that this was already lower-casing DOIs, resulting in some broken metadata getting created. See also: https://github.com/internetarchive/fatcat/issues/83 This is just the first step of mitigation.
* web: fix DOAJ article links (remove trailing slash)Bryan Newbold2021-06-041-1/+1
|
* dblp tests: skip redundant seek(0)Bryan Newbold2021-06-031-6/+1
|
* ingest: swap ingest and file checks, to result in clearer stats/counts of ↵Bryan Newbold2021-06-031-2/+2
| | | | skipping
* ingest: don't accept mag and s2 URLsBryan Newbold2021-06-031-4/+4
|
* bump fuzzycat dependency to 0.1.21Bryan Newbold2021-06-022-20/+18
|
* web: fix spacing for doaj/dblp identifiers in SERPBryan Newbold2021-05-311-1/+1
|
* ingest: don't 'track_total_hits' for ES 7.x count()Bryan Newbold2021-05-311-1/+1
|
* web: bugfix dblp vs. doaj display logicBryan Newbold2021-05-311-1/+1
|
* update fuzzycat to 0.1.20Bryan Newbold2021-05-312-31/+94
|
* makefile: add pylint -E invocation to 'make lint', to match CIBryan Newbold2021-05-251-0/+1
|
* skip pylint on 'assigning-non-slot' warnings in Flask 2.0Bryan Newbold2021-05-251-2/+2
| | | | | | | | | | | | | | The 'permanent' field is still valid to set to a boolean in Flask 2.0; not sure why pylint is unhappy in CI (causing test failures). Don't see any problem running test suite locally. Flask API docs: https://flask.palletsprojects.com/en/2.0.x/api/?highlight=permanent#flask.session.permanent And code (recent master branch): https://github.com/pallets/flask/blob/4240ace59710d86c478111affd4ad6fb4c8cad9e/src/flask/sessions.py#L20
* changelog worker: fix file/fileset typo, caught by lintBryan Newbold2021-05-251-1/+1
| | | | | This would have been resulting in some releases not getting re-indexed into search.
* small python lint fixes (no behavior change)Bryan Newbold2021-05-255-6/+4
|
* bump Flask to 2.x; other deps upgraded automaticallyBryan Newbold2021-05-212-152/+167
|
* ingest: add per-container ingest type overridesBryan Newbold2021-05-212-1/+23
|
* fix arabesque sqlite3 examples to have 14-digit timestampsBryan Newbold2021-05-211-0/+0
|
* arabesque importer: ensure full 14-digit timestampsBryan Newbold2021-05-211-1/+3
|
* Andrew W. Mellon FoundationBryan Newbold2021-05-182-3/+3
|
* Merge branch 'bnewbold-pipenv-cleanup' into 'master'bnewbold2021-04-232-327/+277
|\ | | | | | | | | pipenv cleanup See merge request webgroup/fatcat!104
| * pipenv: re-lock projectBryan Newbold2021-04-191-301/+253
| |
| * pipenv: constrain most package versions to at least majorBryan Newbold2021-04-191-24/+24
| | | | | | | | | | | | | | | | | | Don't have a complete policy with this change, just locking things down a bit more so small package additions and updates don't end up upgrading some small dependency to a major new backwards-incompatible version. Also, correct bs4 -> beautifulsoup4 (bs4 is the import name, not the package name)
| * pipenv: remove unused pg-view and pykafka librariesBryan Newbold2021-04-191-2/+0
| |
* | web: fix edit form style guide linksBryan Newbold2021-04-202-4/+4
|/
* transforms: fix 'display_ame' typoBryan Newbold2021-04-191-2/+2
|
* web: expand release creators in more situationsBryan Newbold2021-04-192-2/+2
|
* fix public API linksBryan Newbold2021-04-151-2/+2
|
* fix 'colected' typosBryan Newbold2021-04-131-1/+1
| | | | Thanks for the catch martin
* prefer contrib.creator.display_name over contrib.raw_nameBryan Newbold2021-04-124-9/+17
| | | | | | | | These will be getting updates from ORCID and are usually more complete and more correct for display, attribution, and search purposes. Might need to tweak fuzzycat code to handle multiple names at the verification stage.
* make dblp tests more robustBryan Newbold2021-04-121-2/+11
| | | | | | These were causing a lot of spurious errors in local development. Not sure these tweaks will entirely fix the problem.
* web: show file size not known, when it isn'tBryan Newbold2021-04-121-0/+2
| | | | This is mostly to prevent showing an empty metadata box
* web: better logic for showing 'save-paper-now' linkBryan Newbold2021-04-121-0/+2
|
* web: include DOI in share-your-paper URL, when possibleBryan Newbold2021-04-121-2/+8
|
* web: consistent public API URLsBryan Newbold2021-04-126-14/+9
|
* web: improve preservation holdings display for containersBryan Newbold2021-04-121-10/+22
|
* web: improve access button HTMLBryan Newbold2021-04-122-3/+2
|
* web: add goatcounter analyticsBryan Newbold2021-04-123-0/+16
| | | | Same setup as scholar.archive.org
* es worker: ensure kafka messages get clearedBryan Newbold2021-04-121-0/+2
|
* es indexing: more 'wip' fixesBryan Newbold2021-04-121-1/+5
|
* ES indexing: skip 'wip' entities with a warningBryan Newbold2021-04-121-11/+16
|
* fatcat_ingest: fix recent lint failureBryan Newbold2021-04-091-1/+1
|
* search: more ES 7.x changes (track total counts)Bryan Newbold2021-04-092-0/+12
|
* container ES index worker: support for querying statusBryan Newbold2021-04-062-5/+37
|
* transform tool: container transform stats lookup supportBryan Newbold2021-04-062-2/+27
|
* ES schema updates: doc_index_ts as a str, not datetimeBryan Newbold2021-04-061-4/+4
| | | | | The schema is a timestamp, but python needs to serialize as JSON, and doesn't do datetime automatically.
* web infra: log to stderrBryan Newbold2021-04-061-2/+4
|
* search container stats: changes to be called from index code pathBryan Newbold2021-04-062-3/+20
| | | | Eg, allowing injection of more config values
* container search schema: preservation stats, new fieldsBryan Newbold2021-04-062-7/+60
| | | | Includes transform code updates and partial test coverage.
* release ES: add discipline fieldBryan Newbold2021-04-061-0/+2
|
generated by cgit v1.2.3 (git 2.39.1) at 2024-11-27 13:16:12 +0000