| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| ... | |
| | |/ |
|
| |/ |
|
| |\
| |
| |
| |
| | |
verify release_stage in ingest importer
See merge request webgroup/fatcat!52
|
| | | |
|
| | | |
|
| |/
|
|
|
|
|
|
|
| |
"span" short for "timespan" to harvest; there may be a better name to
use.
Motivation for this is to work around a pylint erorr that .next() was
not callable. This might be a bug with pylint, but .next() is also a
very generic name.
|
| | |
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Gitlab CI is showing lint errors like:
=================================== FAILURES ===================================
6316 _______________________ [pylint] tests/harvest_state.py ________________________
6317 E: 19,11: hs.next is not callable (not-callable)
6318 E: 33,11: hs.next is not callable (not-callable)
6319 E: 19,11: hs.next is not callable (not-callable)
[...]
this is confusing as we use pipenv with a lock, so I should see the
exact same errors locally.
This commit is a hack to try and fix this and unbreak builds until we
can debug further.
|
| |
|
|
|
| |
It seems to be an inadvertantly ugraded version of pylint saying that
these lines are not-callable.
|
| | |
|
| |\ |
|
| | | |
|
| | | |
|
| | |
| |
| |
| |
| |
| |
| |
| | |
One of these (in ingest importer pipeline) is an actual bug, the others
are just changing the syntax to be more explicit/conservative.
The ingest importer bug seems to have resulted in some bad file match
imports; scale of impact is unknown.
|
| | |
| |
| |
| |
| |
| |
| | |
Until reviewing I didn't realize we were even doing this currently.
Hopefluly has not impacted too many imports, as almost all ingests use
an external identifer, so only those with identifers not in fatcat for
whatever reason.
|
| |\ \
| | |
| | |
| | |
| | | |
search: assume * when q is not set or empty
See merge request webgroup/fatcat!51
|
| | | |
| | |
| | |
| | | |
An example would be a blank search from a container details page.
|
| |\ \ \
| |/ /
|/| |
| | |
| | | |
tweaks to search result pages
See merge request webgroup/fatcat!50
|
| | | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | |
| | | |
This is also back-ported from covid19.fatcat.wiki, though with some more
tweaks on top.
The changes are:
- show original title if available (usually non-English)
- move release_type label to title line suffix, and only show if not a
"paper"
- show publication status and withdrawl as text after the journal title,
not as a label
|
| | |/
| |
| |
| | |
These are back-ported fixes from covid19.fatcat.wiki
|
| |\ \
| | |
| | |
| | |
| | |
| | |
| | | |
fix ident=None broken links
Closes #3
See merge request webgroup/fatcat!49
|
| | |/
| |
| |
| |
| |
| | |
On web interface views for revisions, we had a bunch of broken links
because the ident is "None". This commit fixes these by removing the
links.
|
| | |
| |
| |
| |
| |
| |
| | |
Up to now, we expected the description to be a string or list. Add
handling for int as well.
First appeared: Apr 22 19:58:39.
|
| |\ \
| | |
| | |
| | |
| | |
| | |
| | | |
into 'master'
datacite: fix a raw name constraint violation
See merge request webgroup/fatcat!47
|
| | |/
| |
| |
| |
| |
| |
| | |
It was possible that contribs got added which had no raw name. One
example would be a name consisting of whitespace only.
This fix adds a final check for this case.
|
| | | |
|
| |/
|
|
|
| |
The API fetch update may be needed for old changelog entries in the
kafka feed.
|
| |\
| |
| |
| |
| | |
py37 cleanups
See merge request webgroup/fatcat!44
|
| | | |
|
| | | |
|
| | | |
|
| | |
| |
| |
| |
| | |
We had some pre-3.6 work arounds. Also seems like a reasonable time to
update all depdencies to most recent versions.
|
| |\ \
| | |
| | |
| | |
| | | |
derive changelog worker from release worker
See merge request webgroup/fatcat!43
|
| | |/
| |
| |
| |
| | |
Early versions of changelog entries may not have all the fields
required for the current transform.
|
| | |
| |
| |
| |
| | |
No partial docs (e.g. abstract), too generic components and entries, not
HTML blogs.
|
| |/
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
according to release_rev.release_type, we have 29 values:
fatcat_prod=# select release_type, count(release_type) from release_rev group by release_type;
release_type | count
-------------------+-----------
abstract | 2264
article | 6371076
article-journal | 101083841
article-newspaper | 17062
book | 1676941
chapter | 13914854
component | 58990
dataset | 6860325
editorial | 133573
entry | 1628487
graphic | 1809471
interview | 19898
legal_case | 3581
legislation | 1626
letter | 275119
paper-conference | 6074669
peer_review | 30581
post | 245807
post-weblog | 135
report | 1010699
retraction | 1292
review-book | 96219
software | 316
song | 24027
speech | 4263
standard | 312364
stub | 1036813
thesis | 414397
| 0
(29 rows)
|
| | |
|
| |
|
|
| |
Also updates dependencies.
|
| | |
|
| |\
| |
| |
| |
| | |
beautifulsoup XML parsing: .string vs. .get_text()
See merge request webgroup/fatcat!40
|
| | |
| |
| |
| |
| |
| |
| | |
The primary motivation for this change is that fatcat *requires* a
non-empty title for each release entity. Pubmed/Medline occasionally
indexes just a VenacularTitle with no ArticleTitle for foreign
publications, and currently those records don't end up in fatcat at all.
|
| | | |
|
| | |
| |
| |
| | |
See previous pubmed commit for details.
|
| | |
| |
| |
| |
| |
| |
| |
| |
| |
| |
| | |
Yikes! Apparently when a tag has child tags, .string will return None
instead of all the strings. .get_text() returns all of it:
https://www.crummy.com/software/BeautifulSoup/bs4/doc/#get-text
https://www.crummy.com/software/BeautifulSoup/bs4/doc/#string
I've things like identifiers as .string, when we expect only a single
string inside.
|
| | |
| |
| |
| |
| |
| | |
This goes against what the API docs recommend, but we are currently far
behind on updates and need to catch up. Other than what the docs say,
this seems to be consistent with the behavior we want.
|
| | | |
|
| | | |
|
| |\ \
| |/
|/| |
|
| | |
| |
| |
| | |
Thanks to Martin for suggestion
|
