index
:
fatcat
bnewbold-doaj-article-harvest
bnewbold-elastic-extras
bnewbold-openapi-client-generator-v601
bnewbold-pythonclient-types
bnewbold-redoc
bnewbold-rust-gen-v5
bnewbold-sitemap
bnewbold-ubuntu-jammy
cockroach
confluent-kafka
master
preview
x-attic-auth-other-macaroon-lib
x-attic-camp
x-attic-changelog-export
x-attic-chocula
x-attic-cockroach
x-attic-golang
x-attic-more-importers
x-attic-preview
x-attic-python-rust-hacks
[no description]
summary
refs
log
tree
commit
diff
stats
log msg
author
committer
range
path:
root
/
python
/
fatcat_tools
Commit message (
Expand
)
Author
Age
Files
Lines
*
ES indexing: skip 'wip' entities with a warning
Bryan Newbold
2021-04-12
1
-11
/
+16
*
container ES index worker: support for querying status
Bryan Newbold
2021-04-06
1
-5
/
+32
*
ES schema updates: doc_index_ts as a str, not datetime
Bryan Newbold
2021-04-06
1
-4
/
+4
*
container search schema: preservation stats, new fields
Bryan Newbold
2021-04-06
1
-2
/
+18
*
release ES: add discipline field
Bryan Newbold
2021-04-06
1
-0
/
+2
*
ES schemas: add doc_index_ts to all mappings
Bryan Newbold
2021-04-06
1
-0
/
+4
*
indexing: don't use document names
Bryan Newbold
2021-04-06
1
-14
/
+4
*
datacite: a missing surname should be None, not the empty string
Martin Czygan
2021-04-02
1
-2
/
+1
*
elasticsearch: simple new dblp and doaj fields
Bryan Newbold
2021-01-20
1
-0
/
+4
*
web ingest: terminal URL mismatch as skip, not assert
Bryan Newbold
2020-12-30
1
-1
/
+3
*
dblp release import: skip arxiv_id releases
Bryan Newbold
2020-12-24
1
-0
/
+9
*
normalizer: test for un-versioned arxiv_id
Bryan Newbold
2020-12-24
1
-0
/
+4
*
dblp import: fix arxiv_id typo
Bryan Newbold
2020-12-23
1
-1
/
+1
*
ingest: allow dblp imports
Bryan Newbold
2020-12-23
1
-1
/
+1
*
fuzzy: set 120 second timeout on ES lookups
Bryan Newbold
2020-12-23
1
-1
/
+1
*
dblp: polish HTML scrape/extract pipeline
Bryan Newbold
2020-12-17
1
-0
/
+14
*
dblp: flesh out update code path (especially to add container_id linkage)
Bryan Newbold
2020-12-17
1
-2
/
+6
*
dblp: run fuzzy matching at try_update time (same as DOAJ)
Bryan Newbold
2020-12-17
1
-1
/
+8
*
improve dblp release import
Bryan Newbold
2020-12-17
1
-1
/
+2
*
very simple dblp container importer
Bryan Newbold
2020-12-17
2
-0
/
+145
*
dblp release importer: container_id lookup TSV, and dump JSON mode
Bryan Newbold
2020-12-17
1
-10
/
+66
*
wikidata QID normalize helper
Bryan Newbold
2020-12-17
1
-2
/
+24
*
initial implementation of dblp release importer (in progress)
Bryan Newbold
2020-12-17
2
-0
/
+445
*
add 'lxml' mode for large XML file import, and multi-tags
Bryan Newbold
2020-12-17
1
-15
/
+28
*
add dblp as an ingest source and identifier
Bryan Newbold
2020-12-17
1
-1
/
+2
*
ingest: allow doaj ingest responses
Bryan Newbold
2020-12-17
1
-1
/
+2
*
bug fix: is_preserved should always be bool
Bryan Newbold
2020-12-17
1
-2
/
+2
*
Merge branch 'bnewbold-doaj-fuzzy' into 'master'
bnewbold
2020-12-18
2
-4
/
+71
|
\
|
*
update fuzzy helper to pass 'reason' through to import code
Bryan Newbold
2020-12-17
1
-3
/
+3
|
*
add fuzzy match filtering to DOAJ importer
Bryan Newbold
2020-12-16
1
-2
/
+9
|
*
add fuzzy matching helper to importer base class
Bryan Newbold
2020-12-16
1
-2
/
+62
*
|
entity update worker: treat fileset and webcapture updates like file updates
Bryan Newbold
2020-12-16
1
-3
/
+25
*
|
fix indentation
Bryan Newbold
2020-12-16
1
-2
/
+2
*
|
have release elasticsearch transform count webcaptures and filesets towards p...
Bryan Newbold
2020-12-16
1
-26
/
+57
*
|
small release_to_elasticsearch refactors
Bryan Newbold
2020-12-16
1
-7
/
+12
*
|
refactor release_to_elasticsearch transform
Bryan Newbold
2020-12-16
1
-131
/
+148
|
/
*
html ingest: small fixes to try_update() code path
Bryan Newbold
2020-12-15
1
-5
/
+5
*
HACK: squash intermitent failure of detect_text_lang() test
Bryan Newbold
2020-12-11
1
-1
/
+2
*
langdetect: more text for 'zh' test case
Bryan Newbold
2020-11-20
1
-1
/
+1
*
crossref+datacite: remove confusing early update bail
Bryan Newbold
2020-11-20
2
-4
/
+0
*
doaj: fix update code path (getattr not __dict__)
Bryan Newbold
2020-11-20
1
-4
/
+3
*
DOAJ: handle empty identifier 'id' case
Bryan Newbold
2020-11-20
1
-0
/
+2
*
clean DOI: ban all non-ASCII characters
Bryan Newbold
2020-11-19
1
-1
/
+4
*
normal: handle langdetect of 'zh-cn' (not len=2)
Bryan Newbold
2020-11-19
1
-0
/
+3
*
tweak DOAJ importer class args and default for do_updates
Bryan Newbold
2020-11-19
1
-2
/
+2
*
if a release has DOAJ article id, count as OA
Bryan Newbold
2020-11-19
1
-0
/
+3
*
implement remainder of DOAJ article importer
Bryan Newbold
2020-11-19
1
-57
/
+125
*
handle more non-ASCII DOI cases
Bryan Newbold
2020-11-19
1
-1
/
+3
*
more python normalizers, and move from importer common
Bryan Newbold
2020-11-19
2
-154
/
+326
*
initial implementation of DOAJ importer
Bryan Newbold
2020-11-19
2
-0
/
+290
[next]