aboutsummaryrefslogtreecommitdiffstats
path: root/guide/src
diff options
context:
space:
mode:
Diffstat (limited to 'guide/src')
-rw-r--r--guide/src/entity_fields.md418
1 files changed, 282 insertions, 136 deletions
diff --git a/guide/src/entity_fields.md b/guide/src/entity_fields.md
index 939ec084..29d318dc 100644
--- a/guide/src/entity_fields.md
+++ b/guide/src/entity_fields.md
@@ -6,16 +6,19 @@ All entities have:
The "extra" field is an "escape hatch" to include extra fields not in the
regular schema. It is intended to enable gradual evolution of the schema, as
-well as accommodating niche or field-specific content. That being said,
-reasonable limits should be adhered to.
+well as accommodating niche or field-specific content. Reasonable care should
+be taken with this extra metadata: don't include large text or binary fields,
+hundreds of fields, duplicate metadata, etc.
## Containers
-- `name`: (string, required). The title of the publication, as used in
+- `name` (string, required): The title of the publication, as used in
international indexing services. Eg, "Journal of Important Results". Not
necessarily in the native language, but also not necessarily in English.
- Alternative titles (and translations) can be stored in "extra" metadata
- (TODO: what field?).
+ Alternative titles (and translations) can be stored in "extra" metadata (see
+ below)
+- `container_type` (string): eg, journal vs. conference vs. book series.
+ Controlled vocabulary is TODO.
- `publisher` (string): The name of the publishing organization. Eg, "Society
of Curious Students".
- `issnl` (string): an external identifier, with registration controlled by the
@@ -27,51 +30,148 @@ reasonable limits should be adhered to.
can cause confusion. The ISSN master list is not gratis/public, but the
ISSN-L mapping is.
- `wikidata_qid` (string): external linking identifier to a Wikidata entity.
+
+#### `extra` Fields
+
- `abbrev` (string): a commonly used abbreviation for the publication, as used
in citations, following the [ISO 4]() standard. Eg, "Journal of Polymer
- Science Part A" -> "J. Polym. Sci. A". Alternative abbreviations can be
- stored in "extra" metadata. (TODO: what field?)
+ Science Part A" -> "J. Polym. Sci. A"
- `coden` (string): an external identifier, the [CODEN code](). 6 characters,
all upper-case.
+- `issnp` (string): Print ISSN
+- `issne` (string): Electronic ISSN
+- `default_license` (string, slug): short name (eg, "CC-BY-SA") for the
+ default/recommended license for works published in this container
+- `original_name` (string): native name (if `name` is translated)
+- `platform` (string): hosting platform: OJS, wordpress, scielo, etc
+- `mimetypes` (array of string): formats that this container publishes all works
+ under (eg, 'application/pdf', 'text/html')
+- `first_year` (integer): first year of publication
+- `last_year` (integer): final year of publication (implies that container is no longer active)
+- `languages` (array of strings): ISO codes; the first entry is considered the
+ "primary" language (if that makes sense)
+- `country` (string): ISO abbreviation (two characters) for the country this
+ container is published in
+- `aliases` (array of strings): significant alternative names or abbreviations
+ for this container (not just capitalization/punctuation)
+- `region` (string, slug): continent/world-region (vocabulary is TODO)
+- `discipline` (string, slug): highest-level subject aread (vocabulary is TODO)
+- `urls` (array of strings): known homepage URLs for this container (first in array is default)
+
+Additional fields used in analytics and "curration" tracking:
+
+- `doaj` (object)
+ - `as_of` (string, ISO datetime): datetime of most recent check; if not set,
+ not actually in DOAJ
+ - `seal` (bool): has DOAJ seal
+ - `work_level` (bool): whether work-level publications are registered with DOAJ
+ - `archive` (array of strings): preservation archives
+- `road` (object)
+ - `as_of` (string, ISO datetime): datetime of most recent check; if not set,
+ not actually in ROAD
+- `kbart` (object)
+ - `lockss`, `clockss`, `portico`, `jstor` etc (object)
+ - `year_spans` (array of arrays of integers (pairs)): year spans (inclusive)
+ for which the given archive has preserved this container
+ - `volume_spans` (array of arrays of integers (pairs)): volume spans (inclusive)
+ for which the given archive has preserved this container
+- `sherpa_romeo` (object):
+ - `color` (string): the SHERPA/RoMEO "color" of the publisher of this container
+- `doi`: TODO: include list of prefixes and which (if any) DOI registrar is used
+- `dblp` (object):
+ - `id` (string)
+- `ia` (object): Internet Archive specific fields
+ - `sim` (object): same format as `kbart` preservation above; coverage in microfilm collection
+ - `longtail` (bool): is this considered a "long-tail" open access venue
+
[CODEN]: https://en.wikipedia.org/wiki/CODEN
## Creators
-See ["Human Names"](./style_guide.index##human-names) sub-section of style
-guide.
-
-- `display_name` (string, required): Eg, "Grace Hopper".
-- `given_name` (string): Eg, "Grace".
-- `surname` (string): Eg, "Hooper".
+- `display_name` (string, required): Full name, as will be displayed in user
+ interfaces. Eg, "Grace Hopper"
+- `given_name` (string): Also known as "first name". Eg, "Grace".
+- `surname` (string): Also known as "last name". Eg, "Hooper".
- `orcid` (string): external identifier, as registered with ORCID.
- `wikidata_qid` (string): external linking identifier to a Wikidata entity.
+
+See also ["Human Names"](./style_guide.md##human-names) sub-section of style guide.
## Files
-- `size` (positive, non-zero integer): Eg: 1048576.
-- `sha1` (string): Eg: "f013d66c7f6817d08b7eb2a93e6d0440c1f3e7f8".
-- `md5`: Eg: "d41efcc592d1e40ac13905377399eb9b".
-- `sha256`: Eg: "a77e4c11a57f1d757fca5754a8f83b5d4ece49a2d28596889127c1a2f3f28832".
+- `size` (integer, positive, non-zero): Size of file in bytes. Eg: 1048576.
+- `md5` (string): MD5 hash in lower-case hex. Eg: "d41efcc592d1e40ac13905377399eb9b".
+- `sha1` (string): SHA-1 hash in lower-case hex. Not required, but the most-used of the hashes and should always be included. Eg: "f013d66c7f6817d08b7eb2a93e6d0440c1f3e7f8".
+- `sha256`: SHA-256 hash in lower-case hex. Eg: "a77e4c11a57f1d757fca5754a8f83b5d4ece49a2d28596889127c1a2f3f28832".
- `urls`: An array of "typed" URLs. Order is not meaningful, and may not be
preserved.
- `url` (string, required):
Eg: "https://example.edu/~frau/prcding.pdf".
- `rel` (string, required):
Eg: "webarchive".
-- `mimetype` (string):
- example: "application/pdf"
-- `releases` (array of identifiers): references to `release` entities that this
+- `mimetype` (string): Format of the file. If XML, specific schema can be
+ included after a `+`. Example: "application/pdf"
+- `release_ids` (array of string identifiers): references to `release` entities that this
file represents a manifestation of. Note that a single file can contain
multiple release references (eg, a PDF containing a full issue with many
articles), and that a release will often have multiple files (differing only
by watermarks, or different digitizations of the same printed work, or
- variant MIME/media types of the same published work). See also
- "Work/Release/File Distinctions".
+ variant MIME/media types of the same published work).
+
+## Filesets
+
+Warning: This schema is not yet stable.
+
+- `manifest` (array of objects): each entry represents a file
+ - `path` (string, required): relative path to file (including filename)
+ - `size` (integer, required): in bytes
+ - `md5` (string): MD5 hash in lower-case hex
+ - `sha1` (string): SHA-1 hash in lower-case hex
+ - `sha256` (string): SHA-256 hash in lower-case hex
+ - `extra` (object): any extra metadata about this specific file
+- `urls`: An array of "typed" URLs. Order is not meaningful, and may not be
+ preserved.
+ - `url` (string, required):
+ Eg: "https://example.edu/~frau/prcding.pdf".
+ - `rel` (string, required):
+ Eg: "webarchive".
+- `release_ids` (array of string identifiers): references to `release` entities
+
+## Webcaptures
+
+Warning: This schema is not yet stable.
+
+- `cdx` (array of objects): each entry represents a distinct web resource
+ (URL). First is considered the primary/entry. Roughly aligns with CDXJ schema.
+ - `surt` (string, required): sortable URL format
+ - `timestamp` (string, datetime, required): ISO format, UTC timezone, with
+ `Z` prefix required, with second (or finer) precision. Eg,
+ "2016-09-19T17:20:24Z". Wayback timestamps (like "20160919172024") should
+ be converted naively.
+ - `url` (string, required): full URL
+ - `mimetype` (string): content type of the resource
+ - `status_code` (integer, signed): HTTP status code
+ - `sha1` (string, required): SHA-1 hash in lower-case hex
+ - `sha256` (string): SHA-256 hash in lower-case hex
+- `archive_urls`: An array of "typed" URLs where this snapshot can be found.
+ Can be wayback/memento instances, or direct links to a WARC file containing
+ all the capture resources. Often will only be a single archive. Order is not
+ meaningful, and may not be preserved.
+ - `url` (string, required):
+ Eg: "https://example.edu/~frau/prcding.pdf".
+ - `rel` (string, required): Eg: "wayback" or "warc"
+- `original_url` (string): base URL of the resource. May reference a specific
+ CDX entry, or maybe in normalized form.
+- `timestamp` (string, datetime): same format as CDX line timestamp (UTC, etc).
+ Corresponds to the overall capture timestamp. Can be the earliest of CDX
+ timestamps if that makes sense
+- `release_ids` (array of string identifiers): references to `release` entities
## Releases
-- `title` (required): the title of the release.
+- `title` (string, required): the display title of the release. May include subtitle.
+- `original_title` (string): the full original language title, if `title` is translated
- `work_id` (fatcat identifier; required): the (single) work that this release
is grouped under. If not specified in a creation (`POST`) action, the API
will auto-generate a work.
@@ -80,34 +180,37 @@ guide.
entity.
- `release_type` (string, controlled set): represents the medium or form-factor
of this release; eg, "book" versus "journal article". Not necessarily
- consistent across all releases of a work. See definitions below.
+ the same across all releases of a work. See definitions below.
- `release_status` (string, controlled set): represents the publishing/review
lifecycle status of this particular release of the work. See definitions
below.
-- `release_date` (string, date format): when this release was first made
- publicly available
+- `release_date` (string, ISO date format): when this release was first made
+ publicly available. Blank if only year is known.
+- `release_year` (integer): year when this release was first made
+ publicly available; should match `release_date` if both are known.
- `doi` (string): full DOI number, lower-case. Example: "10.1234/abcde.789".
See the "External Identifiers" section of style guide.
+- `wikidata_qid` (string): external identifier for Wikidata entities. These are
+ integers prefixed with "Q", like "Q4321". Each `release` entity can be
+ associated with at most one Wikidata entity (this field is not an array), and
+ Wikidata entities should be associated with at most a single `release`. In
+ the future it may be possible to associate Wikidata entities with `work`
+ entities instead. See the "External Identifiers" section of style guide.
- `isbn13` (string): external identifier for books. ISBN-9 and other formats
should be converted to canonical ISBN-13. See the "External Identifiers"
section of style guide.
-- `core_id` (string): external identifier for the [CORE] open access
- aggregator. These identifiers are integers, but stored in string format. See
- the "External Identifiers" section of style guide.
- `pmid` (string): external identifier for PubMed database. These are bare
integers, but stored in a string format. See the "External Identifiers"
section of style guide.
- `pmcid` (string): external identifier for PubMed Central database. These are
integers prefixed with "PMC" (upper case), like "PMC4321". See the "External
Identifiers" section of style guide.
-- `wikidata_qid` (string): external identifier for Wikidata entities. These are
- integers prefixed with "Q", like "Q4321". Each `release` entity can be
- associated with at most one Wikidata entity (this field is not an array), and
- Wikidata entities should be associated with at most a single `release`. In
- the future it may be possible to associate Wikidata entities with `work`
- entities instead. See the "External Identifiers" section of style guide.
+- `core_id` (string): external identifier for the [CORE] open access
+ aggregator. These identifiers are integers, but stored in string format. See
+ the "External Identifiers" section of style guide.
- `arxiv_id` (string) external identifier to a (version-specific) [arxiv.org]()
work
+- `jstor_id` (string) external identifier for works in JSTOR
- `volume` (string): optionally, stores the specific volume of a serial
publication this release was published in.
type: string
@@ -121,12 +224,15 @@ guide.
populated if the associated `container` entity has the publisher field set,
though it is acceptable to duplicate, as the publishing entity of a container
may differ over time. Should be set for singleton releases, like books.
-- `language` (string): the primary language used in this particular release of
+- `language` (string, slug): the primary language used in this particular release of
the work. Only a single language can be specified; additional languages can
be stored in "extra" metadata (TODO: which field?). This field should be a
- valid RFC1766/ISO639-1 language code ("with extensions"), aka a controlled
+ valid RFC1766/ISO639 language code (two letters). AKA, a controlled
vocabulary, not a free-form name of the language.
-- `contribs`: an array of authorship and other `creator` contributions to this
+- `license_slug` (string, slug): the license of this release. Usually a
+ creative commons short code (eg, `CC-BY`), though a small number of other
+ short names for publisher-specific licenses are included (TODO: list these).
+- `contribs` (array of objects): an array of authorship and other `creator` contributions to this
release. Contribution fields include:
- `index` (integer, optional): the (zero-indexed) order of this
author. Authorship order has significance in many fields. Non-author
@@ -144,7 +250,7 @@ guide.
vocabulary. TODO: vocabulary needs review.
- `extra` (string): additional context can go here. For example, author
affiliation, "this is the corresponding author", etc.
-- `refs`: an array of references (aka, citations) to other releases. References
+- `refs` (array of ident strings): references (aka, citations) to other releases. References
can only be linked to a specific target release (not a work), though it may
be ambiguous which release of a work is being referenced if the citation is
not specific enough. Reference fields include:
@@ -169,48 +275,85 @@ guide.
- `locator` (string): a more specific reference into the work/release being
cited, for example the page number(s). For web reference, store the URL
in "extra", not here.
+- `abstracts` (array of objects): see below
+ - `sha1` (string, hex, required): reference to the abstract content (string).
+ Example: "3f242a192acc258bdfdb151943419437f440c313"
+ - `content` (string): The abstract raw content itself. Example: `<jats:p>Some
+ abstract thing goes here</jats:p>`
+ - `mimetype` (string): not formally required, but should effectively always get
+ set. `text/plain` if the abstract doesn't have a structured format
+ - `lang` (string, controlled set): the human language this abstract is in. See
+ the `lang` field of release for format and vocabulary.
-Controlled vocabulary for `release_type` is derived from the Crossref `type`
-vocabulary (TODO: should it follow [CSL types](http://docs.citationstyles.org/en/stable/specification.html#appendix-iii-types) instead?):
-
-- `journal-article`
-- `proceedings-article`
-- `monograph`
-- `dissertation`
-- `book` (and `edited-book`, `reference-book`)
-- `book-chapter` (and `book-part`, `book-section`, though much rarer) is
- allowed as these are frequently referenced and read independent of the entire
- book. The data model does not currently support linking a subset of a release
- to an entity representing the entire release. The release/work/file
- distinctions should not be used to group chapters into complete work; a book
- chapter can be it's own work. A paper which is republished as a chapter (eg,
- in a collection, or "edited" book) can have both releases under one work. The
- criteria of whether to "split" a book and have release entities for each
- chapter is whether the chapter has been cited/reference as such.
-- `dissertation`
-- `dataset` (though representation with `file` entities is TBD).
-- `monograph`
+[arxiv.org]: https://arxiv.org
+
+#### `extra` Fields
+
+- `crossref` (object), for extra crossref-specific metadata
+ - `subject` (array of strings) for subject/category of content
+ - `type` (string) raw/original Crossref type
+ - `alternative-id` (array of strings)
+ - `archive` (array of strings), indicating preservation services deposited
+ - `funder` (object/dictionary)
+- `aliases` (array of strings) for additional titles this release might be
+ known by
+- `container_name` (string) if not matched to a container entity
+- `subtitle` (string)
+- `group-title` (string) for releases within an collection/group
+ `release_status` getting updated)
+- `translation_of` (release identifier) if this release is a translation of
+ another (usually under the same work)
+- `withdrawn_data` (string, ISO date format): if this release has been
+ retracted (post-publication) or withdrawn (pre- or post-publication), this is
+ the datetime of that event. Retractions also result in a `retraction` release
+ under the same `work` entity. This is intended to migrate from "extra" to a
+ full release entity field.
+
+#### `release_type` Vocabulary
+
+This vocabulary is based on the
+[CSL types](http://docs.citationstyles.org/en/stable/specification.html#appendix-iii-types),
+with a small number of (proposed) extensions:
+
+- `article-magazine`
+- `article-newspaper`
+- `article-journal`, including pre-prints and working papers
+- `book`
+- `chapter` is allowed as they are frequently referenced and read independent
+ of the entire book. The data model does not currently support linking a
+ subset of a release to an entity representing the entire release. The
+ release/work/file distinctions should not be used to group multiple chapters under
+ a single work; a book chapter can be it's own work. A paper which is
+ republished as a chapter (eg, in a collection, or "edited" book) can have
+ both releases under one work. The criteria of whether to "split" a book and
+ have release entities for each chapter is whether the chapter has been
+ cited/reference as such.
+- `dataset`
+- `entry`, which can be used for generic web resources like question/answer
+ site entries.
+- `entry-encyclopedia`
+- `manuscript`
+- `paper-conference`
+- `patent`
+- `post-weblog` for blog entries
- `report`
-- `standard`
-- `posted-content` is allowed, but may be re-categorized. For crossref, this
- seems to imply a journal article or report which is not published (pre-print)
-- `other` matches Crossref `other` works, which may (and generally should) have
- a more specific type set.
-- `web-post` (custom extension) for blog posts, essays, and other individual
- works on websites
-- `website` (custom extension) for entire web sites and wikis.
-- `presentation` (custom extension) for, eg, slides and recorded conference
- presentations themselves, as distinct from `proceedings-article`
+- `review`, for things like book reviews, not the "literature review" form of `article-journal`
+- `speech` can be used for eg, slides and recorded conference presentations
+ themselves, as distinct from `paper-conference`
+- `thesis`
+- `webpage`
+- `peer_review` (fatcat extension)
+- `software` (fatcat extension)
+- `standard` (fatcat extension)
+- `abstract` (fatcat extension)
- `editorial` (custom extension) for columns, "in this issue", and other
- content published along peer-reviewed content in journals. Can bleed in to
- "other" or "stub"
-- `book-review` (custom extension)
+ content published along peer-reviewed content in journals.
- `letter` for "letters to the editor", "authors respond", and
sub-article-length published content
- `example` (custom extension) for dummy or example releases that have valid
(registered) identifiers. Other metadata does not need to match "canonical"
examples.
-- `stub` (custom extension) for releases which have notable external
+- `stub` (fatcat extension) for releases which have notable external
identifiers, and thus are included "for completeness", but don't seem to
represent a "full work". An example might be a paper that gets an extra DOI
by accident; the primary DOI should be a full release, and the accidental DOI
@@ -228,75 +371,78 @@ vocabulary (TODO: should it follow [CSL types](http://docs.citationstyles.org/en
- "Acknowledgments"
- "Notices"
-Other types from Crossref (such as `component`, `reference-entry`) are valid,
-but are not actively solicited for inclusion, as they are not the current focus
-of the database.
-
-In the future, some types (like `journal`, `proceedings`, and `book-series`)
-will probably be represented as `container` entities. How to represent other
-container-like types (like `report-series` or `book-series`) is TBD.
-
-Controlled vocabulary for `release_status`:
-- `published` for any version of the work that was "formally published", or any
- variant that can be considered a "proof", "camera ready", "archival",
- "version of record" or "definitive" that have no meaningful differences from
- the "published" version. Note that "meaningful" here will need to be
- explored.
-- `corrected` for a version of a work that, after formal publication, has been
- revised and updated. Could be the "version of record".
-- `pre-print`, for versions of a work which have not been submitted for peer
- review or formal publication
-- `post-print`, often a post-peer-review version of a work that does not have
- publisher-supplied copy-editing, typesetting, etc.
-- `draft` in the context of book publication or online content (shouldn't be
- applied to journal articles), is an unpublished, but somehow notable version
- of a work.
-- If blank, indicates status isn't known, and wasn't inferred at creation time.
- Can often be interpreted as `published`.
-
-Controlled vocabulary for `role` field on `contribs`:
-- `author`
-- `translator`
-- `illustrator`
-- `editor`
-- If blank, indicates that type of contribution is not known; this can often be
- interpreted as authorship.
+All other CSL types are also allowed, though they are mostly out of scope:
-Current "extra" fields, flags, and content:
-- `crossref` (object), for extra crossref-specific metadata
- - `subject` (array of strings) for subject/category of content
- - `type` (string) raw/original Crossref type
- - `alternative-id` (array of strings)
- - `archive` (array of strings), indicating preservation services deposited
- - `funder` (object/dictionary)
-- `aliases` (array of strings) for additional titles this release might be
- known by
-- `container_name` (string) if not matched to a container entity
-- `subtitle` (string)
-- `group-title` (string) for releases within an collection/group
-- `is_retracted` (boolean flag) if this work has been retracted (in addition to
- `release_status` getting updated)
-- `translation_of` (release identifier) if this release is a translation of
- another (usually under the same work)
+- `article` (generic; should usually be some other type)
+- `bill`
+- `broadcast`
+- `entry-dictionary`
+- `figure`
+- `graphic`
+- `interview`
+- `legislation`
+- `legal_case`
+- `map`
+- `motion_picture`
+- `musical_score`
+- `pamphlet`
+- `personal_communication`
+- `post`
+- `review-book`
+- `song`
+- `treaty`
+For the purpose of statistics, the following release types are considered
+"papers":
-[arxiv.org]: https://arxiv.org
+- `article-journal`
+- `chapter`
+- `paper-conference`
+- `thesis`
-### Abstracts
+#### `release_status` Vocabulary
-Abstract *contents* (in raw string form) are stored in their own table, and are
-immutable (not editable), but there is release-specific metadata as part of
-`release` entities.
+These roughly follow the [DRIVER](http://web.archive.org/web/20091109125137/http://www2.lse.ac.uk/library/versions/VERSIONS_Toolkit_v1_final.pdf) publication version guidelines, with the addition of a `retracted` status.
-- `sha1` (string, hex, required): reference to the abstract content (string).
- Example: "3f242a192acc258bdfdb151943419437f440c313"
-- `content` (string): The abstract raw content itself. Example: `<jats:p>Some
- abstract thing goes here</jats:p>`
-- `mimetype` (string): not formally required, but should effectively always get
- set. `text/plain` if the abstract doesn't have a structured format
-- `lang` (string, controlled set): the human language this abstract is in. See
- the `lang` field of release for format and vocabulary.
+- `draft` is an early version of a work which is not considered for peer
+ review. Sometimes these are posted to websites or repositories for early
+ comments and feedback.
+- `submitted` is the version that was submitted for publication. Also known as
+ "pre-print", "pre-review", "under review". Note that this doesn't imply that
+ the work was every actually submitted, reviewed, or accepted for publication,
+ just that this is the version that "would be". Most versions in pre-print
+ repositories are likely to have this status.
+- `accepted` is a version that has undergone peer review and accepted for
+ published, but has not gone through any publisher copy editing or
+ re-formatting. Also known as "post-print", "author's manuscript",
+ "publisher's proof".
+- `published` is the version that the publisher distributes. May include minor
+ (gramatical, typographical, broken link, aesthetic) corrections. Also known
+ as "version of record", "final publication version", "archival copy".
+- `updated`: post-publication significant updates (considered a separate release
+ in Fatcat). Also known as "correction" (in the context of either a published
+ "correction notice", or the full new version)
+- `retraction` for post-publication retraction notices (should be a release
+ under the same work as the `published` release)
+
+Note that in the case of a retraction, the original publication does not get
+status `retracted`, only the retraction notice does. The original publication
+does get a `widthdrawn_date` metadata field set.
+
+When blank, indicates status isn't known, and wasn't inferred at creation time.
+Can often be interpreted as `published`, but be careful!
+
+#### `contribs.role` Vocabulary
+
+- `author`
+- `translator`
+- `illustrator`
+- `editor`
+
+If blank, indicates that type of contribution is not known; this can often be
+interpreted as authorship.
## Works
-Works have no field! They just group releases.
+Works have no fields! They just group releases.
+