Commit message (Collapse) | Author | Age | Files | Lines | |
---|---|---|---|---|---|
* | doaj: fix update code path (getattr not __dict__) | Bryan Newbold | 2020-11-20 | 3 | -15/+70 |
| | | | | Also add missing code coverage for update path (disabled by default). | ||||
* | DOAJ: handle empty identifier 'id' case | Bryan Newbold | 2020-11-20 | 1 | -0/+2 |
| | |||||
* | clean DOI: ban all non-ASCII characters | Bryan Newbold | 2020-11-19 | 1 | -1/+4 |
| | | | | | | | I believe this is safe and matches the regex filter in rust (fatcatd). Keep hitting one-off DOIs that were passing through python check, so being more strict from here forward. | ||||
* | normal: handle langdetect of 'zh-cn' (not len=2) | Bryan Newbold | 2020-11-19 | 1 | -0/+3 |
| | |||||
* | update fatcatd rust code for 'oai' external identifier | Bryan Newbold | 2020-11-19 | 4 | -11/+189 |
| | |||||
* | codegen rust schema crate | Bryan Newbold | 2020-11-19 | 6 | -3/+20 |
| | |||||
* | codegen python openapi client | Bryan Newbold | 2020-11-19 | 2 | -4/+36 |
| | |||||
* | schema: also add 'oai' identifer (OAI-PMH) for releases | Bryan Newbold | 2020-11-19 | 2 | -2/+9 |
| | |||||
* | tweak DOAJ importer class args and default for do_updates | Bryan Newbold | 2020-11-19 | 1 | -2/+2 |
| | |||||
* | show DOAJ (and dblp) identifiers in release view | Bryan Newbold | 2020-11-19 | 1 | -1/+7 |
| | |||||
* | if a release has DOAJ article id, count as OA | Bryan Newbold | 2020-11-19 | 1 | -0/+3 |
| | |||||
* | implement remainder of DOAJ article importer | Bryan Newbold | 2020-11-19 | 3 | -68/+168 |
| | |||||
* | handle more non-ASCII DOI cases | Bryan Newbold | 2020-11-19 | 1 | -1/+3 |
| | |||||
* | more python normalizers, and move from importer common | Bryan Newbold | 2020-11-19 | 2 | -154/+326 |
| | | | | | | | | | | | | Moved several normalizer helpers out of fatcat_tools.importers.common to fatcat_tools.normal. Copied language name and country name parser helpers from chocula repository (built on existing pycountry helper library). Have not gone through and refactored other importers to point to these helpers yet; that should be a separate PR when this branch is merged. Current changes are backwards compatible via re-imports. | ||||
* | initial implementation of DOAJ importer | Bryan Newbold | 2020-11-19 | 4 | -0/+387 |
| | | | | Several things to finish implementing and polish. | ||||
* | python API client: resolve warning about '\d' in string | Bryan Newbold | 2020-11-19 | 1 | -2/+2 |
| | |||||
* | rustfmt | Bryan Newbold | 2020-11-19 | 5 | -87/+138 |
| | |||||
* | rust: fatcatd changes for DOAJ+dblp identifiers | Bryan Newbold | 2020-11-19 | 6 | -949/+1062 |
| | |||||
* | codegen rust crate for v0.3.3 | Bryan Newbold | 2020-11-19 | 8 | -227/+244 |
| | |||||
* | codegen python client library for v0.3.3 | Bryan Newbold | 2020-11-19 | 7 | -16/+80 |
| | |||||
* | schema: DOAJ+dblp ext_ids; bump to v0.3.3 | Bryan Newbold | 2020-11-19 | 2 | -1/+25 |
| | |||||
* | ingest and proposal updates | Bryan Newbold | 2020-11-19 | 2 | -0/+45 |
| | |||||
* | Merge branch 'bnewbold-xml-html-ingest' into 'master' | Martin Czygan | 2020-11-19 | 10 | -66/+409 |
|\ | | | | | | | | | HTML webcapture ingest (and XML file ingest) See merge request webgroup/fatcat!88 | ||||
| * | html ingest: actual xhtml mimetype | Bryan Newbold | 2020-11-16 | 1 | -2/+2 |
| | | |||||
| * | ingest tool: support for setting ingest type | Bryan Newbold | 2020-11-06 | 2 | -6/+10 |
| | | |||||
| * | html ingest: remaining implementation | Bryan Newbold | 2020-11-06 | 1 | -22/+19 |
| | | |||||
| * | ingest: fix XML ingest test file | Bryan Newbold | 2020-11-05 | 1 | -1/+1 |
| | | |||||
| * | ingest: progress on HTML ingest | Bryan Newbold | 2020-11-05 | 3 | -16/+74 |
| | | |||||
| * | ingest: initial 'web' worker implementation | Bryan Newbold | 2020-11-05 | 3 | -67/+301 |
| | | |||||
| * | refactor: white/black -> allow/block | Bryan Newbold | 2020-11-05 | 1 | -4/+4 |
| | | |||||
| * | ingest: whitelist -> allowlist | Bryan Newbold | 2020-11-05 | 2 | -6/+6 |
| | | |||||
| * | ingest: tests for basic XML ingest | Bryan Newbold | 2020-11-05 | 2 | -0/+18 |
| | | |||||
| * | ingest: basic checks for ingest_type | Bryan Newbold | 2020-11-05 | 3 | -4/+36 |
|/ | |||||
* | normalizer: filter out a specific non-ASCII character in DOI | Bryan Newbold | 2020-11-04 | 1 | -1/+3 |
| | |||||
* | entity updates: don't ingest JSTOR DOI prefixes | Bryan Newbold | 2020-10-23 | 1 | -0/+2 |
| | |||||
* | Merge branch 'bnewbold-scholar-pipeline' into 'master' | bnewbold | 2020-10-20 | 2 | -2/+26 |
|\ | | | | | | | | | entity updater: new work update feed (ident and changelog metadata only) See merge request webgroup/fatcat!87 | ||||
| * | entity updater: new work update feed (ident and changelog metadata only) | Bryan Newbold | 2020-10-16 | 2 | -2/+26 |
|/ | |||||
* | bulk citation graph workflow proposal | Bryan Newbold | 2020-10-15 | 1 | -0/+160 |
| | |||||
* | Merge branch 'bnewbold-web-tweaks-20201013' into 'master' | Martin Czygan | 2020-10-14 | 4 | -12/+19 |
|\ | | | | | | | | | web coverage tweaks 20201013 See merge request webgroup/fatcat!86 | ||||
| * | container coverage: add keeper link and KBART holdings list | Bryan Newbold | 2020-10-13 | 1 | -0/+11 |
| | | |||||
| * | release view: remove abiguous OA status indicator | Bryan Newbold | 2020-10-13 | 1 | -4/+0 |
| | | |||||
| * | container view: fix non-OA empty box | Bryan Newbold | 2020-10-13 | 1 | -3/+3 |
| | | |||||
| * | coverage: show counts and fraction in tooltip of coverage bars | Bryan Newbold | 2020-10-13 | 1 | -5/+5 |
|/ | |||||
* | update database/table stats | Bryan Newbold | 2020-10-12 | 2 | -0/+48 |
| | |||||
* | chocula importer: small tweaks to update behavior | Bryan Newbold | 2020-10-08 | 1 | -8/+6 |
| | |||||
* | elastic transform: more preservation keepers | Bryan Newbold | 2020-10-08 | 1 | -1/+2 |
| | |||||
* | update 'contributing' page in guide | Bryan Newbold | 2020-10-02 | 5 | -17/+176 |
| | |||||
* | update README | Bryan Newbold | 2020-10-01 | 1 | -41/+51 |
| | |||||
* | updates to CHANGELOG | Bryan Newbold | 2020-10-01 | 1 | -0/+19 |
| | |||||
* | more metadata cleanup task notes | Bryan Newbold | 2020-10-01 | 1 | -0/+7 |
| |