aboutsummaryrefslogtreecommitdiffstats
path: root/skate
Commit message (Collapse)AuthorAgeFilesLines
* reduce: temp bug fix for line cutterMartin Czygan2021-07-132-32/+61
| | | | | | | | we wanted to trim whitespace at one point, because values contained the separator values; however this breaks with empty values; move back to not trimming values except for the newline, when requesting the last value; moving forward, we need to clean or reject dirty values or use a different delimiter
* v0.1.38Martin Czygan2021-07-131-1/+1
|
* reduce: small tweaksMartin Czygan2021-07-132-6/+7
|
* fix typoMartin Czygan2021-07-131-1/+1
|
* wip: csl loggingMartin Czygan2021-07-131-1/+1
|
* update docsMartin Czygan2021-07-131-1/+7
|
* reduce/schema: add cslMartin Czygan2021-07-133-5/+70
|
* wiki: include lang in encoded page titleMartin Czygan2021-07-132-8/+18
|
* reduce: add todoMartin Czygan2021-07-131-1/+3
|
* separate slugify functionsMartin Czygan2021-07-134-28/+39
|
* mock out time.Now for testsMartin Czygan2021-07-134-1034/+1041
|
* reduce: log broken line onlyMartin Czygan2021-07-101-1/+1
|
* reduce: add key and indexed ts for exact matchesMartin Czygan2021-07-101-0/+2
|
* batch: drop loggingMartin Czygan2021-07-101-4/+0
|
* batch: log batch sizeMartin Czygan2021-07-101-1/+1
|
* reduce: short circuit large groupsMartin Czygan2021-07-101-2/+12
| | | | | | | | we saw a jump in memory usage, and it may be related to groups with thousands of elements; e.g. maybe some weird string, that appears too many times as key, e.g. 123/test; as a first measure, we sort circuit further batching; other mitigiation may to be limit groups size completely
* schema: prefer isbn13Martin Czygan2021-07-101-1/+5
|
* schema: render isbn as wellMartin Czygan2021-07-101-1/+7
|
* reduce: ol, fuzzy, w/ unstructuredMartin Czygan2021-07-101-1/+1
|
* schema: add testMartin Czygan2021-07-102-0/+20
|
* schema: flesh our unstructured renderingMartin Czygan2021-07-102-0/+56
|
* release to unstructured stubMartin Czygan2021-07-103-2/+84
|
* reduce: open library id tweaksMartin Czygan2021-07-101-5/+27
|
* reduce: tweak wiki brefMartin Czygan2021-07-101-4/+5
|
* reduce: filter out duplicate wiki linksMartin Czygan2021-07-101-0/+8
|
* wiki: use lowercase base32 of page titleMartin Czygan2021-07-091-2/+3
| | | | * mostly case insensitive, same case as ident
* reduce: use a base64 encoded title as keyMartin Czygan2021-07-091-1/+7
|
* wiki: cleanup redundant checkMartin Czygan2021-07-091-1/+1
|
* wiki: tweak whitespace handlingMartin Czygan2021-07-091-1/+7
|
* wiki: more aggressive whitespace cleanupMartin Czygan2021-07-091-1/+2
|
* wiki: try a bit more cleanupMartin Czygan2021-07-091-1/+5
|
* wiki: verify doiMartin Czygan2021-07-091-1/+1
|
* unstructured: cleanup obsolete regexMartin Czygan2021-07-091-9/+3
|
* reduce: wiki doc in column 3Martin Czygan2021-07-091-1/+1
|
* tests: sync verify test dataMartin Czygan2021-07-096-0/+176
|
* wiki: flip doi and page title columnMartin Czygan2021-07-091-3/+3
|
* reduce: move batch sizeMartin Czygan2021-07-092-9/+9
|
* reduce: prepare command line helpMartin Czygan2021-07-081-0/+12
|
* update docsMartin Czygan2021-07-081-3/+3
|
* reduce: set default batch sizeMartin Czygan2021-07-081-6/+8
|
* simplify importsMartin Czygan2021-07-089-9/+9
|
* reduce: separate batch callsMartin Czygan2021-07-082-20/+25
|
* update docsMartin Czygan2021-07-071-3/+8
|
* skate: no need for aliasMartin Czygan2021-07-071-1/+1
|
* util: cleanup encoderMartin Czygan2021-07-061-19/+0
|
* reduce: remove log lineMartin Czygan2021-07-061-1/+0
|
* reduce: move to threaded versionsMartin Czygan2021-07-061-25/+30
|
* add resource usage noteMartin Czygan2021-07-061-1/+1
|
* wip: improve reduce performanceMartin Czygan2021-07-061-50/+8
|
* batcher: do not pass struct fieldsMartin Czygan2021-07-051-4/+4
|
generated by cgit v1.2.3 (git 2.39.1) at 2024-11-22 17:51:31 +0000