diff options
Diffstat (limited to 'guide/src')
| -rw-r--r-- | guide/src/entity_fields.md | 418 |
1 files changed, 282 insertions, 136 deletions
diff --git a/guide/src/entity_fields.md b/guide/src/entity_fields.md index 939ec084..29d318dc 100644 --- a/guide/src/entity_fields.md +++ b/guide/src/entity_fields.md @@ -6,16 +6,19 @@ All entities have: The "extra" field is an "escape hatch" to include extra fields not in the regular schema. It is intended to enable gradual evolution of the schema, as -well as accommodating niche or field-specific content. That being said, -reasonable limits should be adhered to. +well as accommodating niche or field-specific content. Reasonable care should +be taken with this extra metadata: don't include large text or binary fields, +hundreds of fields, duplicate metadata, etc. ## Containers -- `name`: (string, required). The title of the publication, as used in +- `name` (string, required): The title of the publication, as used in international indexing services. Eg, "Journal of Important Results". Not necessarily in the native language, but also not necessarily in English. - Alternative titles (and translations) can be stored in "extra" metadata - (TODO: what field?). + Alternative titles (and translations) can be stored in "extra" metadata (see + below) +- `container_type` (string): eg, journal vs. conference vs. book series. + Controlled vocabulary is TODO. - `publisher` (string): The name of the publishing organization. Eg, "Society of Curious Students". - `issnl` (string): an external identifier, with registration controlled by the @@ -27,51 +30,148 @@ reasonable limits should be adhered to. can cause confusion. The ISSN master list is not gratis/public, but the ISSN-L mapping is. - `wikidata_qid` (string): external linking identifier to a Wikidata entity. + +#### `extra` Fields + - `abbrev` (string): a commonly used abbreviation for the publication, as used in citations, following the [ISO 4]() standard. Eg, "Journal of Polymer - Science Part A" -> "J. Polym. Sci. A". Alternative abbreviations can be - stored in "extra" metadata. (TODO: what field?) + Science Part A" -> "J. Polym. Sci. A" - `coden` (string): an external identifier, the [CODEN code](). 6 characters, all upper-case. +- `issnp` (string): Print ISSN +- `issne` (string): Electronic ISSN +- `default_license` (string, slug): short name (eg, "CC-BY-SA") for the + default/recommended license for works published in this container +- `original_name` (string): native name (if `name` is translated) +- `platform` (string): hosting platform: OJS, wordpress, scielo, etc +- `mimetypes` (array of string): formats that this container publishes all works + under (eg, 'application/pdf', 'text/html') +- `first_year` (integer): first year of publication +- `last_year` (integer): final year of publication (implies that container is no longer active) +- `languages` (array of strings): ISO codes; the first entry is considered the + "primary" language (if that makes sense) +- `country` (string): ISO abbreviation (two characters) for the country this + container is published in +- `aliases` (array of strings): significant alternative names or abbreviations + for this container (not just capitalization/punctuation) +- `region` (string, slug): continent/world-region (vocabulary is TODO) +- `discipline` (string, slug): highest-level subject aread (vocabulary is TODO) +- `urls` (array of strings): known homepage URLs for this container (first in array is default) + +Additional fields used in analytics and "curration" tracking: + +- `doaj` (object) + - `as_of` (string, ISO datetime): datetime of most recent check; if not set, + not actually in DOAJ + - `seal` (bool): has DOAJ seal + - `work_level` (bool): whether work-level publications are registered with DOAJ + - `archive` (array of strings): preservation archives +- `road` (object) + - `as_of` (string, ISO datetime): datetime of most recent check; if not set, + not actually in ROAD +- `kbart` (object) + - `lockss`, `clockss`, `portico`, `jstor` etc (object) + - `year_spans` (array of arrays of integers (pairs)): year spans (inclusive) + for which the given archive has preserved this container + - `volume_spans` (array of arrays of integers (pairs)): volume spans (inclusive) + for which the given archive has preserved this container +- `sherpa_romeo` (object): + - `color` (string): the SHERPA/RoMEO "color" of the publisher of this container +- `doi`: TODO: include list of prefixes and which (if any) DOI registrar is used +- `dblp` (object): + - `id` (string) +- `ia` (object): Internet Archive specific fields + - `sim` (object): same format as `kbart` preservation above; coverage in microfilm collection + - `longtail` (bool): is this considered a "long-tail" open access venue + [CODEN]: https://en.wikipedia.org/wiki/CODEN ## Creators -See ["Human Names"](./style_guide.index##human-names) sub-section of style -guide. - -- `display_name` (string, required): Eg, "Grace Hopper". -- `given_name` (string): Eg, "Grace". -- `surname` (string): Eg, "Hooper". +- `display_name` (string, required): Full name, as will be displayed in user + interfaces. Eg, "Grace Hopper" +- `given_name` (string): Also known as "first name". Eg, "Grace". +- `surname` (string): Also known as "last name". Eg, "Hooper". - `orcid` (string): external identifier, as registered with ORCID. - `wikidata_qid` (string): external linking identifier to a Wikidata entity. + +See also ["Human Names"](./style_guide.md##human-names) sub-section of style guide. ## Files -- `size` (positive, non-zero integer): Eg: 1048576. -- `sha1` (string): Eg: "f013d66c7f6817d08b7eb2a93e6d0440c1f3e7f8". -- `md5`: Eg: "d41efcc592d1e40ac13905377399eb9b". -- `sha256`: Eg: "a77e4c11a57f1d757fca5754a8f83b5d4ece49a2d28596889127c1a2f3f28832". +- `size` (integer, positive, non-zero): Size of file in bytes. Eg: 1048576. +- `md5` (string): MD5 hash in lower-case hex. Eg: "d41efcc592d1e40ac13905377399eb9b". +- `sha1` (string): SHA-1 hash in lower-case hex. Not required, but the most-used of the hashes and should always be included. Eg: "f013d66c7f6817d08b7eb2a93e6d0440c1f3e7f8". +- `sha256`: SHA-256 hash in lower-case hex. Eg: "a77e4c11a57f1d757fca5754a8f83b5d4ece49a2d28596889127c1a2f3f28832". - `urls`: An array of "typed" URLs. Order is not meaningful, and may not be preserved. - `url` (string, required): Eg: "https://example.edu/~frau/prcding.pdf". - `rel` (string, required): Eg: "webarchive". -- `mimetype` (string): - example: "application/pdf" -- `releases` (array of identifiers): references to `release` entities that this +- `mimetype` (string): Format of the file. If XML, specific schema can be + included after a `+`. Example: "application/pdf" +- `release_ids` (array of string identifiers): references to `release` entities that this file represents a manifestation of. Note that a single file can contain multiple release references (eg, a PDF containing a full issue with many articles), and that a release will often have multiple files (differing only by watermarks, or different digitizations of the same printed work, or - variant MIME/media types of the same published work). See also - "Work/Release/File Distinctions". + variant MIME/media types of the same published work). + +## Filesets + +Warning: This schema is not yet stable. + +- `manifest` (array of objects): each entry represents a file + - `path` (string, required): relative path to file (including filename) + - `size` (integer, required): in bytes + - `md5` (string): MD5 hash in lower-case hex + - `sha1` (string): SHA-1 hash in lower-case hex + - `sha256` (string): SHA-256 hash in lower-case hex + - `extra` (object): any extra metadata about this specific file +- `urls`: An array of "typed" URLs. Order is not meaningful, and may not be + preserved. + - `url` (string, required): + Eg: "https://example.edu/~frau/prcding.pdf". + - `rel` (string, required): + Eg: "webarchive". +- `release_ids` (array of string identifiers): references to `release` entities + +## Webcaptures + +Warning: This schema is not yet stable. + +- `cdx` (array of objects): each entry represents a distinct web resource + (URL). First is considered the primary/entry. Roughly aligns with CDXJ schema. + - `surt` (string, required): sortable URL format + - `timestamp` (string, datetime, required): ISO format, UTC timezone, with + `Z` prefix required, with second (or finer) precision. Eg, + "2016-09-19T17:20:24Z". Wayback timestamps (like "20160919172024") should + be converted naively. + - `url` (string, required): full URL + - `mimetype` (string): content type of the resource + - `status_code` (integer, signed): HTTP status code + - `sha1` (string, required): SHA-1 hash in lower-case hex + - `sha256` (string): SHA-256 hash in lower-case hex +- `archive_urls`: An array of "typed" URLs where this snapshot can be found. + Can be wayback/memento instances, or direct links to a WARC file containing + all the capture resources. Often will only be a single archive. Order is not + meaningful, and may not be preserved. + - `url` (string, required): + Eg: "https://example.edu/~frau/prcding.pdf". + - `rel` (string, required): Eg: "wayback" or "warc" +- `original_url` (string): base URL of the resource. May reference a specific + CDX entry, or maybe in normalized form. +- `timestamp` (string, datetime): same format as CDX line timestamp (UTC, etc). + Corresponds to the overall capture timestamp. Can be the earliest of CDX + timestamps if that makes sense +- `release_ids` (array of string identifiers): references to `release` entities ## Releases -- `title` (required): the title of the release. +- `title` (string, required): the display title of the release. May include subtitle. +- `original_title` (string): the full original language title, if `title` is translated - `work_id` (fatcat identifier; required): the (single) work that this release is grouped under. If not specified in a creation (`POST`) action, the API will auto-generate a work. @@ -80,34 +180,37 @@ guide. entity. - `release_type` (string, controlled set): represents the medium or form-factor of this release; eg, "book" versus "journal article". Not necessarily - consistent across all releases of a work. See definitions below. + the same across all releases of a work. See definitions below. - `release_status` (string, controlled set): represents the publishing/review lifecycle status of this particular release of the work. See definitions below. -- `release_date` (string, date format): when this release was first made - publicly available +- `release_date` (string, ISO date format): when this release was first made + publicly available. Blank if only year is known. +- `release_year` (integer): year when this release was first made + publicly available; should match `release_date` if both are known. - `doi` (string): full DOI number, lower-case. Example: "10.1234/abcde.789". See the "External Identifiers" section of style guide. +- `wikidata_qid` (string): external identifier for Wikidata entities. These are + integers prefixed with "Q", like "Q4321". Each `release` entity can be + associated with at most one Wikidata entity (this field is not an array), and + Wikidata entities should be associated with at most a single `release`. In + the future it may be possible to associate Wikidata entities with `work` + entities instead. See the "External Identifiers" section of style guide. - `isbn13` (string): external identifier for books. ISBN-9 and other formats should be converted to canonical ISBN-13. See the "External Identifiers" section of style guide. -- `core_id` (string): external identifier for the [CORE] open access - aggregator. These identifiers are integers, but stored in string format. See - the "External Identifiers" section of style guide. - `pmid` (string): external identifier for PubMed database. These are bare integers, but stored in a string format. See the "External Identifiers" section of style guide. - `pmcid` (string): external identifier for PubMed Central database. These are integers prefixed with "PMC" (upper case), like "PMC4321". See the "External Identifiers" section of style guide. -- `wikidata_qid` (string): external identifier for Wikidata entities. These are - integers prefixed with "Q", like "Q4321". Each `release` entity can be - associated with at most one Wikidata entity (this field is not an array), and - Wikidata entities should be associated with at most a single `release`. In - the future it may be possible to associate Wikidata entities with `work` - entities instead. See the "External Identifiers" section of style guide. +- `core_id` (string): external identifier for the [CORE] open access + aggregator. These identifiers are integers, but stored in string format. See + the "External Identifiers" section of style guide. - `arxiv_id` (string) external identifier to a (version-specific) [arxiv.org]() work +- `jstor_id` (string) external identifier for works in JSTOR - `volume` (string): optionally, stores the specific volume of a serial publication this release was published in. type: string @@ -121,12 +224,15 @@ guide. populated if the associated `container` entity has the publisher field set, though it is acceptable to duplicate, as the publishing entity of a container may differ over time. Should be set for singleton releases, like books. -- `language` (string): the primary language used in this particular release of +- `language` (string, slug): the primary language used in this particular release of the work. Only a single language can be specified; additional languages can be stored in "extra" metadata (TODO: which field?). This field should be a - valid RFC1766/ISO639-1 language code ("with extensions"), aka a controlled + valid RFC1766/ISO639 language code (two letters). AKA, a controlled vocabulary, not a free-form name of the language. -- `contribs`: an array of authorship and other `creator` contributions to this +- `license_slug` (string, slug): the license of this release. Usually a + creative commons short code (eg, `CC-BY`), though a small number of other + short names for publisher-specific licenses are included (TODO: list these). +- `contribs` (array of objects): an array of authorship and other `creator` contributions to this release. Contribution fields include: - `index` (integer, optional): the (zero-indexed) order of this author. Authorship order has significance in many fields. Non-author @@ -144,7 +250,7 @@ guide. vocabulary. TODO: vocabulary needs review. - `extra` (string): additional context can go here. For example, author affiliation, "this is the corresponding author", etc. -- `refs`: an array of references (aka, citations) to other releases. References +- `refs` (array of ident strings): references (aka, citations) to other releases. References can only be linked to a specific target release (not a work), though it may be ambiguous which release of a work is being referenced if the citation is not specific enough. Reference fields include: @@ -169,48 +275,85 @@ guide. - `locator` (string): a more specific reference into the work/release being cited, for example the page number(s). For web reference, store the URL in "extra", not here. +- `abstracts` (array of objects): see below + - `sha1` (string, hex, required): reference to the abstract content (string). + Example: "3f242a192acc258bdfdb151943419437f440c313" + - `content` (string): The abstract raw content itself. Example: `<jats:p>Some + abstract thing goes here</jats:p>` + - `mimetype` (string): not formally required, but should effectively always get + set. `text/plain` if the abstract doesn't have a structured format + - `lang` (string, controlled set): the human language this abstract is in. See + the `lang` field of release for format and vocabulary. -Controlled vocabulary for `release_type` is derived from the Crossref `type` -vocabulary (TODO: should it follow [CSL types](http://docs.citationstyles.org/en/stable/specification.html#appendix-iii-types) instead?): - -- `journal-article` -- `proceedings-article` -- `monograph` -- `dissertation` -- `book` (and `edited-book`, `reference-book`) -- `book-chapter` (and `book-part`, `book-section`, though much rarer) is - allowed as these are frequently referenced and read independent of the entire - book. The data model does not currently support linking a subset of a release - to an entity representing the entire release. The release/work/file - distinctions should not be used to group chapters into complete work; a book - chapter can be it's own work. A paper which is republished as a chapter (eg, - in a collection, or "edited" book) can have both releases under one work. The - criteria of whether to "split" a book and have release entities for each - chapter is whether the chapter has been cited/reference as such. -- `dissertation` -- `dataset` (though representation with `file` entities is TBD). -- `monograph` +[arxiv.org]: https://arxiv.org + +#### `extra` Fields + +- `crossref` (object), for extra crossref-specific metadata + - `subject` (array of strings) for subject/category of content + - `type` (string) raw/original Crossref type + - `alternative-id` (array of strings) + - `archive` (array of strings), indicating preservation services deposited + - `funder` (object/dictionary) +- `aliases` (array of strings) for additional titles this release might be + known by +- `container_name` (string) if not matched to a container entity +- `subtitle` (string) +- `group-title` (string) for releases within an collection/group + `release_status` getting updated) +- `translation_of` (release identifier) if this release is a translation of + another (usually under the same work) +- `withdrawn_data` (string, ISO date format): if this release has been + retracted (post-publication) or withdrawn (pre- or post-publication), this is + the datetime of that event. Retractions also result in a `retraction` release + under the same `work` entity. This is intended to migrate from "extra" to a + full release entity field. + +#### `release_type` Vocabulary + +This vocabulary is based on the +[CSL types](http://docs.citationstyles.org/en/stable/specification.html#appendix-iii-types), +with a small number of (proposed) extensions: + +- `article-magazine` +- `article-newspaper` +- `article-journal`, including pre-prints and working papers +- `book` +- `chapter` is allowed as they are frequently referenced and read independent + of the entire book. The data model does not currently support linking a + subset of a release to an entity representing the entire release. The + release/work/file distinctions should not be used to group multiple chapters under + a single work; a book chapter can be it's own work. A paper which is + republished as a chapter (eg, in a collection, or "edited" book) can have + both releases under one work. The criteria of whether to "split" a book and + have release entities for each chapter is whether the chapter has been + cited/reference as such. +- `dataset` +- `entry`, which can be used for generic web resources like question/answer + site entries. +- `entry-encyclopedia` +- `manuscript` +- `paper-conference` +- `patent` +- `post-weblog` for blog entries - `report` -- `standard` -- `posted-content` is allowed, but may be re-categorized. For crossref, this - seems to imply a journal article or report which is not published (pre-print) -- `other` matches Crossref `other` works, which may (and generally should) have - a more specific type set. -- `web-post` (custom extension) for blog posts, essays, and other individual - works on websites -- `website` (custom extension) for entire web sites and wikis. -- `presentation` (custom extension) for, eg, slides and recorded conference - presentations themselves, as distinct from `proceedings-article` +- `review`, for things like book reviews, not the "literature review" form of `article-journal` +- `speech` can be used for eg, slides and recorded conference presentations + themselves, as distinct from `paper-conference` +- `thesis` +- `webpage` +- `peer_review` (fatcat extension) +- `software` (fatcat extension) +- `standard` (fatcat extension) +- `abstract` (fatcat extension) - `editorial` (custom extension) for columns, "in this issue", and other - content published along peer-reviewed content in journals. Can bleed in to - "other" or "stub" -- `book-review` (custom extension) + content published along peer-reviewed content in journals. - `letter` for "letters to the editor", "authors respond", and sub-article-length published content - `example` (custom extension) for dummy or example releases that have valid (registered) identifiers. Other metadata does not need to match "canonical" examples. -- `stub` (custom extension) for releases which have notable external +- `stub` (fatcat extension) for releases which have notable external identifiers, and thus are included "for completeness", but don't seem to represent a "full work". An example might be a paper that gets an extra DOI by accident; the primary DOI should be a full release, and the accidental DOI @@ -228,75 +371,78 @@ vocabulary (TODO: should it follow [CSL types](http://docs.citationstyles.org/en - "Acknowledgments" - "Notices" -Other types from Crossref (such as `component`, `reference-entry`) are valid, -but are not actively solicited for inclusion, as they are not the current focus -of the database. - -In the future, some types (like `journal`, `proceedings`, and `book-series`) -will probably be represented as `container` entities. How to represent other -container-like types (like `report-series` or `book-series`) is TBD. - -Controlled vocabulary for `release_status`: -- `published` for any version of the work that was "formally published", or any - variant that can be considered a "proof", "camera ready", "archival", - "version of record" or "definitive" that have no meaningful differences from - the "published" version. Note that "meaningful" here will need to be - explored. -- `corrected` for a version of a work that, after formal publication, has been - revised and updated. Could be the "version of record". -- `pre-print`, for versions of a work which have not been submitted for peer - review or formal publication -- `post-print`, often a post-peer-review version of a work that does not have - publisher-supplied copy-editing, typesetting, etc. -- `draft` in the context of book publication or online content (shouldn't be - applied to journal articles), is an unpublished, but somehow notable version - of a work. -- If blank, indicates status isn't known, and wasn't inferred at creation time. - Can often be interpreted as `published`. - -Controlled vocabulary for `role` field on `contribs`: -- `author` -- `translator` -- `illustrator` -- `editor` -- If blank, indicates that type of contribution is not known; this can often be - interpreted as authorship. +All other CSL types are also allowed, though they are mostly out of scope: -Current "extra" fields, flags, and content: -- `crossref` (object), for extra crossref-specific metadata - - `subject` (array of strings) for subject/category of content - - `type` (string) raw/original Crossref type - - `alternative-id` (array of strings) - - `archive` (array of strings), indicating preservation services deposited - - `funder` (object/dictionary) -- `aliases` (array of strings) for additional titles this release might be - known by -- `container_name` (string) if not matched to a container entity -- `subtitle` (string) -- `group-title` (string) for releases within an collection/group -- `is_retracted` (boolean flag) if this work has been retracted (in addition to - `release_status` getting updated) -- `translation_of` (release identifier) if this release is a translation of - another (usually under the same work) +- `article` (generic; should usually be some other type) +- `bill` +- `broadcast` +- `entry-dictionary` +- `figure` +- `graphic` +- `interview` +- `legislation` +- `legal_case` +- `map` +- `motion_picture` +- `musical_score` +- `pamphlet` +- `personal_communication` +- `post` +- `review-book` +- `song` +- `treaty` +For the purpose of statistics, the following release types are considered +"papers": -[arxiv.org]: https://arxiv.org +- `article-journal` +- `chapter` +- `paper-conference` +- `thesis` -### Abstracts +#### `release_status` Vocabulary -Abstract *contents* (in raw string form) are stored in their own table, and are -immutable (not editable), but there is release-specific metadata as part of -`release` entities. +These roughly follow the [DRIVER](http://web.archive.org/web/20091109125137/http://www2.lse.ac.uk/library/versions/VERSIONS_Toolkit_v1_final.pdf) publication version guidelines, with the addition of a `retracted` status. -- `sha1` (string, hex, required): reference to the abstract content (string). - Example: "3f242a192acc258bdfdb151943419437f440c313" -- `content` (string): The abstract raw content itself. Example: `<jats:p>Some - abstract thing goes here</jats:p>` -- `mimetype` (string): not formally required, but should effectively always get - set. `text/plain` if the abstract doesn't have a structured format -- `lang` (string, controlled set): the human language this abstract is in. See - the `lang` field of release for format and vocabulary. +- `draft` is an early version of a work which is not considered for peer + review. Sometimes these are posted to websites or repositories for early + comments and feedback. +- `submitted` is the version that was submitted for publication. Also known as + "pre-print", "pre-review", "under review". Note that this doesn't imply that + the work was every actually submitted, reviewed, or accepted for publication, + just that this is the version that "would be". Most versions in pre-print + repositories are likely to have this status. +- `accepted` is a version that has undergone peer review and accepted for + published, but has not gone through any publisher copy editing or + re-formatting. Also known as "post-print", "author's manuscript", + "publisher's proof". +- `published` is the version that the publisher distributes. May include minor + (gramatical, typographical, broken link, aesthetic) corrections. Also known + as "version of record", "final publication version", "archival copy". +- `updated`: post-publication significant updates (considered a separate release + in Fatcat). Also known as "correction" (in the context of either a published + "correction notice", or the full new version) +- `retraction` for post-publication retraction notices (should be a release + under the same work as the `published` release) + +Note that in the case of a retraction, the original publication does not get +status `retracted`, only the retraction notice does. The original publication +does get a `widthdrawn_date` metadata field set. + +When blank, indicates status isn't known, and wasn't inferred at creation time. +Can often be interpreted as `published`, but be careful! + +#### `contribs.role` Vocabulary + +- `author` +- `translator` +- `illustrator` +- `editor` + +If blank, indicates that type of contribution is not known; this can often be +interpreted as authorship. ## Works -Works have no field! They just group releases. +Works have no fields! They just group releases. + |