summaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
* control shadow preservation display with a flagBryan Newbold2020-07-305-19/+80
|
* coverage search: 'recent' endpoint test (minimal)Bryan Newbold2020-07-301-1/+32
|
* release search: fix fulltext_only checkbox (from query object)Bryan Newbold2020-07-302-1/+2
|
* search: 'recent' mode for coverage searchBryan Newbold2020-07-304-7/+134
|
* expand test coverage of new preservation viewsBryan Newbold2020-07-301-15/+122
|
* first iteration of flexible search coverageBryan Newbold2020-07-305-2/+148
|
* make some ES agg methods work on general queriesBryan Newbold2020-07-302-32/+58
| | | | | | | Eg, instead of container-specific year histogram, have the histogram fetch function take a ReleaseQuery. This is in preparation for more generic coverage visualization pages.
* refactor coverage tests/mocksBryan Newbold2020-07-305-39/+80
|
* move container _stats enrichment to routesBryan Newbold2020-07-302-9/+4
| | | | | These were being added to many views which didn't need them (eg, metadata, editing views)
* coverage test mock fixesBryan Newbold2020-07-301-14/+51
|
* web: move random article enrichment to special caseBryan Newbold2020-07-302-6/+5
| | | | | Instead of always enriching container entities with random articles, only do so for the primary container view.
* lint coverage changes (so far)Bryan Newbold2020-07-303-16/+4
|
* update container view statsBryan Newbold2020-07-305-85/+158
| | | | | | - show release type counts - new-style preservation status (single bar) - show release_type preservation break down on coverage page
* preservation coverage updates (first round)Bryan Newbold2020-07-304-32/+326
| | | | | | | | | | - new by-year chart with stacked histograms of all 4 preservation statuses - new-style single progress bar showing overall preservation status - new by-volume query and chart Old endpoints are left as-is, with the intention of having them "deprecated" for some time span until entirely removing them.
* experimentally update semantic-ui to fomantic-ui 2.8.6Bryan Newbold2020-07-301-4/+4
| | | | | | | | | | fomatic-ui is a "community fork" which has had more active recent feature additions, though i'm not sure whether the quality has been as good as the main semantic-ui project. the proximal motivation to upgrade/switch is to get multi-bar progress bars, to display overall preservation status in a single bar as a fraction.
* include new-style preservation+release_type aggs in container statsBryan Newbold2020-07-302-5/+47
|
* coverage UI changes proposalBryan Newbold2020-07-301-0/+80
|
* graphics: handle empty coverage yearsBryan Newbold2020-07-301-7/+8
|
* add regression test for broken container coverageBryan Newbold2020-07-302-57/+98
| | | | also shuffle around search/coverage test files
* Merge branch 'bnewbold-search-improvements' into 'master'bnewbold2020-07-3111-107/+208
|\ | | | | | | | | search and error page improvements See merge request webgroup/fatcat!72
| * update front-page countsBryan Newbold2020-07-281-3/+3
| |
| * switch SERP stage coloring to brown uppercaseBryan Newbold2020-07-281-13/+11
| | | | | | | | To match fatcat-scholor, where this scheme has been successful.
| * error handling: use 400 page with error passed instead of flash()Bryan Newbold2020-07-283-21/+23
| |
| * remove some spurious print statementsBryan Newbold2020-07-282-4/+1
| |
| * convert some API errors from abort() to raiseBryan Newbold2020-07-282-14/+14
| |
| * generic API error pageBryan Newbold2020-07-282-0/+50
| | | | | | | | | | | | | | | | | | | | This error handler and view page currently works much better than the "flash()" infrastructure built-in to flask, which uses cookies and mostly does not work with our views and layouts. Would like to gradually migrate almost all API errors in the web interface to just raising errors that get rendered on an error page, instead of calling `abort(ae.status)`.
| * search: catch ES errors and display betterBryan Newbold2020-07-285-20/+46
| |
| * refactor search macros into new fileBryan Newbold2020-07-284-45/+72
| |
| * include container_id as a query boost termBryan Newbold2020-07-281-0/+1
|/
* Merge branch 'martin-guide-bulk-link-publicdate-desc' into 'master'bnewbold2020-07-271-1/+1
|\ | | | | | | | | guide: link to bulk exports, most recent first See merge request webgroup/fatcat!70
| * guide: link to bulk exports, most recent firstMartin Czygan2020-07-261-1/+1
|/
* Merge branch 'bnewbold-es-refactor' into 'master'bnewbold2020-07-248-373/+506
|\ | | | | | | | | web ES search refactor See merge request webgroup/fatcat!68
| * re-order search params to satisfy pylintBryan Newbold2020-07-241-6/+6
| | | | | | | | | | | | Moved all the request_cache=True param calls to just before ES request exectuation. The former ordering "just worked", but pylint didn't like it, and I suspose it was not as idiomatic as it should have been.
| * small lint fixesBryan Newbold2020-07-242-3/+1
| |
| * finish backend refactoring of search codeBryan Newbold2020-07-242-135/+185
| |
| * update web_search tests to mock ES clientBryan Newbold2020-07-242-45/+47
| | | | | | | | | | | | Instead of using 'responses' mock of 'requests' library. Tried using 'elasticmock' helper but it didn't work.
| * refactor release and container searchBryan Newbold2020-07-246-136/+235
| | | | | | | | | | | | | | | | | | | | Based on fatcat-scholar refactoring. This doesn't include refactoring of stats, aggregates, or histograms yet, just the direct queries. Don't have any test coverage yet; intend to try elasticmock or figuring out how to ingest mock JSON results directly.
| * web search: fix pylint errorBryan Newbold2020-07-241-2/+2
| |
| * WIP: refactoring search to use elasticsearch-dslBryan Newbold2020-07-242-153/+137
|/
* Merge branch 'bnewbold-more-lint-fixes' into 'master'Martin Czygan2020-07-2414-34/+26
|\ | | | | | | | | more lint fixes See merge request webgroup/fatcat!69
| * fix issnl typo in pubmedBryan Newbold2020-07-231-1/+1
| | | | | | | | | | | | | | | | | | | | Oh no! This bug may actually have had significant negative impact on metadata in fatcat, in terms of missing container_id associations with pubmed entities. There are about 500k release entities with a PMID but no container_id. Of those, 89k have at least a container_name. Unclear how many would have matched to ISSN-L and thus to a container.
| * remove isascii() work around definition in importers/datacite.pyBryan Newbold2020-07-231-7/+1
| | | | | | | | We are python3.7 now, so this isn't needed.
| * simple lint (flake8) fixes over python codebaseBryan Newbold2020-07-237-19/+18
| | | | | | | | | | | | These should not have any behavior changes, though a number of exception catches are now more general, and there may be long-tail exceptions getting thrown in these statements.
| * fix actual typo in tests (caught by lint)Bryan Newbold2020-07-231-2/+2
| |
| * simple lint (flake8) fixes in testsBryan Newbold2020-07-235-5/+4
| | | | | | | | | | | | The pytest fixture syntax interacts weirdly with flake8 tests, so ignore the "redefinition" and "unused variable" errors more carefully for .py files under ./tests/
* | Merge branch 'bnewbold-preservation-year-offset' into 'master'bnewbold2020-07-242-0/+55
|\ \ | |/ |/| | | | | preservation year offset See merge request webgroup/fatcat!67
| * simplify in_kbart check statementBryan Newbold2020-07-231-1/+1
| | | | | | | | Thanks @martin
| * make in_kbart transform inclusive of last yearBryan Newbold2020-07-232-0/+55
|/ | | | | | | | | | | | | | | | | Frequently when looking at preservation coverage of journals, the current year shows as "un-preserved" when in fact there is robust KBART (keepers, eg CLOCKSS/Portico) coverage. This is partially because we don't update containers with KBART year spans very frequently (which is on us), and partially because KBART reports are often a bit out of day (eg, doesn't show coverage for the current year. For that matter, they probably take a few months to update the previous year as well, but that is a larger time span to fudge over. This patch means we will count Portico/LOCKSS/etc coverage for "last year" to count as coverage of publications dated "this year". Note that for this to be effective/correct, it is assumed that we will update containers with coverage year spans at least once a year, and that we will re-index all releases at least once a year.
* example bad MAG matchBryan Newbold2020-07-231-0/+6
|
* update table/database size statsBryan Newbold2020-07-222-0/+48
|
generated by cgit v1.2.3 (git 2.39.1) at 2024-11-27 18:23:48 +0000