index
:
fatcat
bnewbold-doaj-article-harvest
bnewbold-elastic-extras
bnewbold-openapi-client-generator-v601
bnewbold-pythonclient-types
bnewbold-redoc
bnewbold-rust-gen-v5
bnewbold-sitemap
bnewbold-ubuntu-jammy
cockroach
confluent-kafka
master
preview
x-attic-auth-other-macaroon-lib
x-attic-camp
x-attic-changelog-export
x-attic-chocula
x-attic-cockroach
x-attic-golang
x-attic-more-importers
x-attic-preview
x-attic-python-rust-hacks
[no description]
about
summary
refs
log
tree
commit
diff
stats
log msg
author
committer
range
path:
root
/
python
/
fatcat_tools
Commit message (
Expand
)
Author
Age
Files
Lines
...
*
ingest: allow dblp imports
Bryan Newbold
2020-12-23
1
-1
/
+1
*
fuzzy: set 120 second timeout on ES lookups
Bryan Newbold
2020-12-23
1
-1
/
+1
*
dblp: polish HTML scrape/extract pipeline
Bryan Newbold
2020-12-17
1
-0
/
+14
*
dblp: flesh out update code path (especially to add container_id linkage)
Bryan Newbold
2020-12-17
1
-2
/
+6
*
dblp: run fuzzy matching at try_update time (same as DOAJ)
Bryan Newbold
2020-12-17
1
-1
/
+8
*
improve dblp release import
Bryan Newbold
2020-12-17
1
-1
/
+2
*
very simple dblp container importer
Bryan Newbold
2020-12-17
2
-0
/
+145
*
dblp release importer: container_id lookup TSV, and dump JSON mode
Bryan Newbold
2020-12-17
1
-10
/
+66
*
wikidata QID normalize helper
Bryan Newbold
2020-12-17
1
-2
/
+24
*
initial implementation of dblp release importer (in progress)
Bryan Newbold
2020-12-17
2
-0
/
+445
*
add 'lxml' mode for large XML file import, and multi-tags
Bryan Newbold
2020-12-17
1
-15
/
+28
*
add dblp as an ingest source and identifier
Bryan Newbold
2020-12-17
1
-1
/
+2
*
ingest: allow doaj ingest responses
Bryan Newbold
2020-12-17
1
-1
/
+2
*
bug fix: is_preserved should always be bool
Bryan Newbold
2020-12-17
1
-2
/
+2
*
Merge branch 'bnewbold-doaj-fuzzy' into 'master'
bnewbold
2020-12-18
2
-4
/
+71
|
\
|
*
update fuzzy helper to pass 'reason' through to import code
Bryan Newbold
2020-12-17
1
-3
/
+3
|
*
add fuzzy match filtering to DOAJ importer
Bryan Newbold
2020-12-16
1
-2
/
+9
|
*
add fuzzy matching helper to importer base class
Bryan Newbold
2020-12-16
1
-2
/
+62
*
|
entity update worker: treat fileset and webcapture updates like file updates
Bryan Newbold
2020-12-16
1
-3
/
+25
*
|
fix indentation
Bryan Newbold
2020-12-16
1
-2
/
+2
*
|
have release elasticsearch transform count webcaptures and filesets towards p...
Bryan Newbold
2020-12-16
1
-26
/
+57
*
|
small release_to_elasticsearch refactors
Bryan Newbold
2020-12-16
1
-7
/
+12
*
|
refactor release_to_elasticsearch transform
Bryan Newbold
2020-12-16
1
-131
/
+148
|
/
*
html ingest: small fixes to try_update() code path
Bryan Newbold
2020-12-15
1
-5
/
+5
*
HACK: squash intermitent failure of detect_text_lang() test
Bryan Newbold
2020-12-11
1
-1
/
+2
*
langdetect: more text for 'zh' test case
Bryan Newbold
2020-11-20
1
-1
/
+1
*
crossref+datacite: remove confusing early update bail
Bryan Newbold
2020-11-20
2
-4
/
+0
*
doaj: fix update code path (getattr not __dict__)
Bryan Newbold
2020-11-20
1
-4
/
+3
*
DOAJ: handle empty identifier 'id' case
Bryan Newbold
2020-11-20
1
-0
/
+2
*
clean DOI: ban all non-ASCII characters
Bryan Newbold
2020-11-19
1
-1
/
+4
*
normal: handle langdetect of 'zh-cn' (not len=2)
Bryan Newbold
2020-11-19
1
-0
/
+3
*
tweak DOAJ importer class args and default for do_updates
Bryan Newbold
2020-11-19
1
-2
/
+2
*
if a release has DOAJ article id, count as OA
Bryan Newbold
2020-11-19
1
-0
/
+3
*
implement remainder of DOAJ article importer
Bryan Newbold
2020-11-19
1
-57
/
+125
*
handle more non-ASCII DOI cases
Bryan Newbold
2020-11-19
1
-1
/
+3
*
more python normalizers, and move from importer common
Bryan Newbold
2020-11-19
2
-154
/
+326
*
initial implementation of DOAJ importer
Bryan Newbold
2020-11-19
2
-0
/
+290
*
html ingest: actual xhtml mimetype
Bryan Newbold
2020-11-16
1
-2
/
+2
*
ingest tool: support for setting ingest type
Bryan Newbold
2020-11-06
1
-6
/
+6
*
html ingest: remaining implementation
Bryan Newbold
2020-11-06
1
-22
/
+19
*
ingest: progress on HTML ingest
Bryan Newbold
2020-11-05
1
-14
/
+30
*
ingest: initial 'web' worker implementation
Bryan Newbold
2020-11-05
2
-67
/
+259
*
refactor: white/black -> allow/block
Bryan Newbold
2020-11-05
1
-4
/
+4
*
ingest: whitelist -> allowlist
Bryan Newbold
2020-11-05
1
-3
/
+3
*
ingest: basic checks for ingest_type
Bryan Newbold
2020-11-05
1
-3
/
+29
*
normalizer: filter out a specific non-ASCII character in DOI
Bryan Newbold
2020-11-04
1
-1
/
+3
*
entity updates: don't ingest JSTOR DOI prefixes
Bryan Newbold
2020-10-23
1
-0
/
+2
*
entity updater: new work update feed (ident and changelog metadata only)
Bryan Newbold
2020-10-16
1
-2
/
+24
*
chocula importer: small tweaks to update behavior
Bryan Newbold
2020-10-08
1
-8
/
+6
*
elastic transform: more preservation keepers
Bryan Newbold
2020-10-08
1
-1
/
+2
[prev]
[next]