| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
| |
Includes a trivial test and transform, but not any workers or doc
updates.
|
| |
|
|
|
|
| |
From martin, thanks.
|
|
|
|
|
|
|
| |
`ingest_oa_only` behavior, and other filters, now handled in the entity
update worker, instead of in the transform function.
Also add a DOI prefix blocklist feature.
|
|
|
|
| |
Refactoring to move this filter elsewhere
|
| |
|
| |
|
|
|
|
|
|
|
|
|
| |
Technically, [...] DOI names may incorporate any printable characters
from the Universal Character Set (UCS-2), of ISO/IEC 10646, which is the
character set defined by Unicode (https://www.doi.org/doi_handbook/2_Numbering.html#2.5.1).
For mostly QA reasons, we currently treat a DOI with an "en dash" as
invalid.
|
| |
|
|
|
|
|
|
| |
Should be backwards compatible with old ingest results.
Fixed a bug with glutton ident detection.
|
| |
|
| |
|
| |
|
| |
|
| |
|
| |
|
| |
|
| |
|
|\
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Pipfile.lock is broken.
* martin-datacite-import: (68 commits)
datacite: pass in doi into factored out method
datacite: reformat test cases and use jq . --sort-keys
datacite: factor out contributor handling
datacite: catch type mismatch in language detection
datacite: adjust tests for release_month
datacite: name extra.month, extra.release_month
datacite: mark additional files as stub
datacite: CCDC are entries, mostly
datacite: use more specific release_type, if possible
datacite: ignore certain names
datacite: over 3% records have the same title: stub
datacite: fill a few more release_type gaps
datacite: adding datacite-specific extra metadata
datacite: apply pylint suggestions
datacite: fix typos
datacite: set release_stage to published by default
datacite: month field should be top-level
datacite: include month in extra
datacite: indicate mismatched file in test
datacite: clean abstracts, use unknown value tokens
...
|
| | |
|
| |
| |
| |
| |
| |
| |
| | |
Use values from:
* attributes.creators[]
* attributes.contributors[]
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
The GBIF (https://www.gbif.org/) deposits most records under the titles:
* 599243 GBIF Occurrence Download
* 41176 Occurrence Download
Mark them as "stub" for the moment
(https://guide.fatcat.wiki/entity_release.html#release_type-vocabulary).
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
* citeproc: http://docs.citationstyles.org/en/stable/specification.html#appendix-iii-types
* resourceTypeGeneral: https://schema.datacite.org/meta/kernel-4.0/doc/DataCite-MetadataKernel_v4.0.pdf#page=32
* resourceType: uncontrolled, over 170000 distinct values, frequent:
null, Dataset, JournalArticle, PGRFA Material, Journal Article,
Dataset/UNITE Species Hypothesis, ...
General frequency:
* "attributes.types": 18210075,
* "attributes.types.ris": 18058890,
* "attributes.types.bibtex": 18058888,
* "attributes.types.citeproc": 18058890,
* "attributes.types.schemaOrg": 18058929,
* "attributes.types.resourceType": 12737988,
* "attributes.types.resourceTypeGeneral": 16576139,
|
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
* attributes.metadataVersion
* attributes.schemaVersion
* attributes.version (source dependent values, follows suggestions in
https://schema.datacite.org/meta/kernel-4.3/doc/DataCite-MetadataKernel_v4.3.pdf#page=26,
but values vary)
Furthermore:
* attributes.types.resourceTypeGeneral
* attributes.types.resourceType
|
| | |
|
| | |
|
| |
| |
| |
| |
| |
| | |
Set to `None` only if there is no publisher yet.
Docs: https://support.datacite.org/docs/doi-states
|
| | |
|
| |
| |
| |
| |
| | |
> include release_month as a top-level extra field [...] to
auto-populate the schema field from that
|
| |
| |
| |
| |
| |
| |
| |
| | |
Datacite defines placeholders for unknown values:
* https://support.datacite.org/docs/schema-values-unknown-information-v43
Clean abstracts.
|
| | |
|
| | |
|
| |
| |
| |
| |
| |
| | |
> always include extra values for the respective DOI registrars
(datacite, crossref, jalc), even if they are empty ({}), to be used as a
flag so we know which DOI registrar supplied the metadata.
|
| | |
|
| |
| |
| |
| | |
As [...] we will soon add support for release_month field in the release schema.
|
| | |
|
| | |
|
| |
| |
| |
| | |
Estimated time for a single call is in the order of 50ms.
|
| |
| |
| |
| |
| |
| |
| |
| |
| | |
> The convention for display_name and raw_name is to be how the name
would normally be printed, not in index form (surname comma given_name).
So we might need to un-encode names like "Tricart, Pierre".
Use an additional `index_form_to_display_name` function to convert index
from to display form, heuristically.
|
| | |
|
| | |
|
| | |
|
| | |
|
| | |
|