aboutsummaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
...
| * Add basic pagination to search resultsMartin Czygan2019-11-084-14/+67
| | | | | | | | | | | | | | | | | | | | | | | | The "deep paging problem" imposes some limit, which currently is a hardcoded default value, `deep_page_limit=2000` in `do_search`. Elasticsearch can be configured, too: > Note that from + size can not be more than the index.max_result_window index setting, which defaults to 10,000. -- https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-body.html#request-body-search-from-size
* | web: catch MacaroonInitExceptionBryan Newbold2019-11-121-0/+4
| | | | | | | | | | Caught one of these in sentry. Probably due to a crawler? Or typing gibberish in the token form.
* | design notes for a larger databaseBryan Newbold2019-11-121-0/+81
| |
* | old proposals for 'next' schema updateBryan Newbold2019-11-121-0/+38
| |
* | crossref patch bulk importBryan Newbold2019-11-122-0/+63
| |
* | Merge branch 'martin-python-readme-es-note' into 'master'bnewbold2019-11-081-0/+5
|\ \ | | | | | | | | | | | | mention elasticsearch empty index setup See merge request webgroup/fatcat!3
| * | mention elasticsearch empty index setupMartin Czygan2019-11-081-0/+5
| |/ | | | | | | | | | | When setting up with the defaults, all works fine, except that the web search will try to access a local elasticsearch. Mention in README, how to create empty indices.
* | crossref: accurate blank title countsBryan Newbold2019-11-051-0/+1
| |
* | fix crossref component testBryan Newbold2019-11-041-1/+1
| |
* | TODO idea: 'first seen'Bryan Newbold2019-11-041-0/+1
| |
* | crossref: component typeBryan Newbold2019-11-041-1/+3
| |
* | add 'component' as a release_typeBryan Newbold2019-11-042-0/+3
| |
* | crossref: count why skip happenedBryan Newbold2019-11-041-1/+7
| | | | | | | | | | | | Might skip based on release type (eg container, not a paper/release), or missing title, or other reasons. Over 7 million DOIs are getting skipped, curious why.
* | crossref: don't skip on short/null subtitleBryan Newbold2019-11-041-1/+1
|/ | | | This was a bug. Should only set subtitle black, not skip the import.
* note file fixup pushed in prodBryan Newbold2019-10-092-1/+64
|
* move corpus changes to 'notes/bulk_edits'Bryan Newbold2019-10-083-0/+285
|
* commit file cleaner testsBryan Newbold2019-10-081-0/+58
|
* file cleanup tweaks to actually runBryan Newbold2019-10-082-5/+4
|
* refactor duplicated b32_hex function in importersBryan Newbold2019-10-083-21/+11
|
* dict wrapper for entity_from_json()Bryan Newbold2019-10-082-3/+7
|
* new cleanup python tool/frameworkBryan Newbold2019-10-085-0/+300
|
* CHANGELOG entry for previous commitBryan Newbold2019-10-031-0/+6
|
* redirect direct entity underscore linksBryan Newbold2019-10-032-0/+30
|
* export raw affiliation strings for analysisBryan Newbold2019-10-031-0/+17
|
* update rust README with fatcat_test db creation noteBryan Newbold2019-10-031-1/+4
|
* update rust README re: opensslBryan Newbold2019-10-011-17/+1
| | | | | | | | | I believe an openssl library is still required locally, but with the SSL/TLS removal it now doesn't matter if it is OpenSSL 1.0 or 1.1. This is only a temporary work-around. When we update rust code generation, we will need to revisit these changes. The current version of swagger-rs still depends on HTTPS and OpenSSL 1.0 (via dependencies).
* entirely remove unused https flag to fatcatdBryan Newbold2019-09-291-15/+6
|
* cargo update fatcat rust after openssl removalBryan Newbold2019-09-291-76/+32
|
* remove 'client' and hyper-openssl options from fatcat-openapi rust crateBryan Newbold2019-09-291-3/+6
|
* webface: extra <br> in container lookup linksBryan Newbold2019-09-211-1/+1
|
* remove duplicate style ref in container edit viewBryan Newbold2019-09-201-5/+0
|
* review/fix all confluent-kafka produce codeBryan Newbold2019-09-206-27/+75
|
* small fixes to confluent-kafka importers/workersBryan Newbold2019-09-208-26/+69
| | | | | | | | - decrease default changelog pipeline to 5.0sec - fix missing KafkaException harvester imports - more confluent-kafka tweaks - updates to kafka consumer configs - bump elastic updates consumergroup (again)
* update Pipfile.lock after confluent-kafka rebaseBryan Newbold2019-09-201-1/+33
|
* convert pipeline workers from pykafka to confluent-kafkaBryan Newbold2019-09-203-125/+230
|
* small kafka tweaks for robustnessBryan Newbold2019-09-202-0/+5
|
* convert importers to confluent-kafka libraryBryan Newbold2019-09-202-21/+74
|
* bump max message size to ~20 MBytesBryan Newbold2019-09-202-0/+2
|
* fixes to confluent-kafka harvestersBryan Newbold2019-09-203-20/+21
|
* docker-compose: kafka 2.0, and -dev topic namesBryan Newbold2019-09-201-3/+2
|
* first draft harvesters using confluent-kafkaBryan Newbold2019-09-203-48/+104
|
* make default kafka env 'dev', not 'qa'Bryan Newbold2019-09-202-4/+4
|
* add confluent-kafka library (to replace pykafka)Bryan Newbold2019-09-201-0/+1
|
* guide: remove Recurse mention from CoCBryan Newbold2019-09-201-3/+2
|
* CHANGELOG note of python codegen changeBryan Newbold2019-09-191-0/+4
|
* fix another python codegen auth contamination bugBryan Newbold2019-09-182-5/+37
| | | | | Seems to be the classic one where a dict as a default arg gets mutated then reused across instances. Blech.
* python codegen: don't clobber README every timeBryan Newbold2019-09-182-1/+2
|
* python codegen with new openapi-generator toolBryan Newbold2019-09-1879-5689/+6824
|
* update python codegen script with new generatorBryan Newbold2019-09-181-58/+6
|
* spec: invalid to have an exmple hereBryan Newbold2019-09-181-1/+0
|
generated by cgit v1.2.3 (git 2.39.1) at 2024-12-21 07:09:19 +0000