summaryrefslogtreecommitdiffstats
path: root/python
Commit message (Collapse)AuthorAgeFilesLines
* add ingest request transform (and test)Bryan Newbold2019-11-153-1/+68
|
* Merge branch 'martin-search-results-pagination' into 'master'Martin Czygan2019-11-156-20/+82
|\ | | | | | | | | Add basic pagination to search results See merge request webgroup/fatcat!4
| * address test issueMartin Czygan2019-11-151-2/+3
| |
| * adjust search test case for new wordingMartin Czygan2019-11-141-2/+2
| | | | | | | | > "Showing top " -> "Showing first "
| * gray out inactive navigation linksMartin Czygan2019-11-141-2/+2
| | | | | | | | | | | | | | | | As per [this issue](https://github.com/Semantic-Org/Semantic-UI/issues/1885#issuecomment-77619519), text colors are not supported in semantic ui. To not move text too much, gray out inactive links.
| * move pagination into macrosMartin Czygan2019-11-143-43/+51
| | | | | | | | | | | | | | | | | | Two new macros: * top_results(found) * bottom_results(found) wip: move pagination into macro
| * Add basic pagination to search resultsMartin Czygan2019-11-084-14/+67
| | | | | | | | | | | | | | | | | | | | | | | | The "deep paging problem" imposes some limit, which currently is a hardcoded default value, `deep_page_limit=2000` in `do_search`. Elasticsearch can be configured, too: > Note that from + size can not be more than the index.max_result_window index setting, which defaults to 10,000. -- https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-body.html#request-body-search-from-size
* | web: catch MacaroonInitExceptionBryan Newbold2019-11-121-0/+4
| | | | | | | | | | Caught one of these in sentry. Probably due to a crawler? Or typing gibberish in the token form.
* | Merge branch 'martin-python-readme-es-note' into 'master'bnewbold2019-11-081-0/+5
|\ \ | | | | | | | | | | | | mention elasticsearch empty index setup See merge request webgroup/fatcat!3
| * | mention elasticsearch empty index setupMartin Czygan2019-11-081-0/+5
| |/ | | | | | | | | | | When setting up with the defaults, all works fine, except that the web search will try to access a local elasticsearch. Mention in README, how to create empty indices.
* | crossref: accurate blank title countsBryan Newbold2019-11-051-0/+1
| |
* | fix crossref component testBryan Newbold2019-11-041-1/+1
| |
* | crossref: component typeBryan Newbold2019-11-041-1/+3
| |
* | crossref: count why skip happenedBryan Newbold2019-11-041-1/+7
| | | | | | | | | | | | Might skip based on release type (eg container, not a paper/release), or missing title, or other reasons. Over 7 million DOIs are getting skipped, curious why.
* | crossref: don't skip on short/null subtitleBryan Newbold2019-11-041-1/+1
|/ | | | This was a bug. Should only set subtitle black, not skip the import.
* commit file cleaner testsBryan Newbold2019-10-081-0/+58
|
* file cleanup tweaks to actually runBryan Newbold2019-10-082-5/+4
|
* refactor duplicated b32_hex function in importersBryan Newbold2019-10-083-21/+11
|
* dict wrapper for entity_from_json()Bryan Newbold2019-10-082-3/+7
|
* new cleanup python tool/frameworkBryan Newbold2019-10-085-0/+300
|
* redirect direct entity underscore linksBryan Newbold2019-10-032-0/+30
|
* webface: extra <br> in container lookup linksBryan Newbold2019-09-211-1/+1
|
* remove duplicate style ref in container edit viewBryan Newbold2019-09-201-5/+0
|
* review/fix all confluent-kafka produce codeBryan Newbold2019-09-206-27/+75
|
* small fixes to confluent-kafka importers/workersBryan Newbold2019-09-208-26/+69
| | | | | | | | - decrease default changelog pipeline to 5.0sec - fix missing KafkaException harvester imports - more confluent-kafka tweaks - updates to kafka consumer configs - bump elastic updates consumergroup (again)
* update Pipfile.lock after confluent-kafka rebaseBryan Newbold2019-09-201-1/+33
|
* convert pipeline workers from pykafka to confluent-kafkaBryan Newbold2019-09-203-125/+230
|
* small kafka tweaks for robustnessBryan Newbold2019-09-202-0/+5
|
* convert importers to confluent-kafka libraryBryan Newbold2019-09-202-21/+74
|
* bump max message size to ~20 MBytesBryan Newbold2019-09-202-0/+2
|
* fixes to confluent-kafka harvestersBryan Newbold2019-09-203-20/+21
|
* first draft harvesters using confluent-kafkaBryan Newbold2019-09-203-48/+104
|
* make default kafka env 'dev', not 'qa'Bryan Newbold2019-09-202-4/+4
|
* add confluent-kafka library (to replace pykafka)Bryan Newbold2019-09-201-0/+1
|
* handle more external identifiers in pythonBryan Newbold2019-09-182-14/+101
| | | | | This makes it possible to, eg, past an arxiv identifier or SHA-1 hash in the general search box and do a quick lookup.
* webface: fix duration_seconds parsingBryan Newbold2019-09-181-1/+1
|
* add guide editing links to edit forms and signup messageBryan Newbold2019-09-185-5/+26
|
* python webface impl token generationBryan Newbold2019-09-184-1/+85
|
* slightly less annoying 'flash' message headerBryan Newbold2019-09-181-1/+1
|
* remove '@' from archive.org identBryan Newbold2019-09-171-1/+1
|
* IA auth: use itemname not screenname for usernameBryan Newbold2019-09-171-1/+1
| | | | | | | Have run in to several issues with IA screenname being invalid fatcat usernames (eg, containing whitespace). This probably won't catch all such issues, but hopefully most of them.
* skip test_crossref_importer_huge() by defaultBryan Newbold2019-09-131-0/+1
|
* fix coverage of python client libraryBryan Newbold2019-09-131-1/+1
|
* small tweaks to coverage pagesBryan Newbold2019-09-062-1/+2
|
* add python README ref to .env fileBryan Newbold2019-09-051-0/+1
| | | | | For running tests. Perhaps a better solution is setting sane defaults in code? But want to require, eg, secret salts.
* update all other mentions of python client libBryan Newbold2019-09-054-8/+8
|
* refactor all python source for client lib nameBryan Newbold2019-09-0558-229/+229
|
* small fixes to release viewBryan Newbold2019-09-051-14/+24
| | | | | - full width container metadata table accordion - fix "Published in by blah" cases
* fix Importer editgroup_extra pass-throughBryan Newbold2019-09-051-2/+1
|
* container type: journal display as 'Scholarly Journal'Bryan Newbold2019-09-051-1/+1
|
generated by cgit v1.2.3 (git 2.39.1) at 2024-12-14 13:31:49 +0000