index
:
fatcat
bnewbold-doaj-article-harvest
bnewbold-elastic-extras
bnewbold-openapi-client-generator-v601
bnewbold-pythonclient-types
bnewbold-redoc
bnewbold-rust-gen-v5
bnewbold-sitemap
bnewbold-ubuntu-jammy
cockroach
confluent-kafka
master
preview
x-attic-auth-other-macaroon-lib
x-attic-camp
x-attic-changelog-export
x-attic-chocula
x-attic-cockroach
x-attic-golang
x-attic-more-importers
x-attic-preview
x-attic-python-rust-hacks
[no description]
summary
refs
log
tree
commit
diff
stats
log msg
author
committer
range
path:
root
/
python
/
fatcat_tools
Commit message (
Expand
)
Author
Age
Files
Lines
...
*
|
ingest: always try some lancet journals
Bryan Newbold
2020-03-19
1
-0
/
+3
*
|
Merge branch 'martin-kafka-bs4-import' into 'master'
Martin Czygan
2020-03-10
5
-22
/
+318
|
\
\
|
|
/
|
/
|
|
*
common: use smaller batch size since XML parsing may be slow
Martin Czygan
2020-03-10
1
-1
/
+1
|
*
pubmed: log to stderr
Martin Czygan
2020-03-10
1
-1
/
+1
|
*
pubmed: move mapping generation out of fetch_date
Martin Czygan
2020-03-10
1
-7
/
+8
|
*
harvest: fix imports from HarvestPubmedWorker cleanup
Martin Czygan
2020-03-10
1
-2
/
+2
|
*
pubmed: citations is a bit more precise
Martin Czygan
2020-03-09
1
-1
/
+1
|
*
pubmed: we sync from FTP
Martin Czygan
2020-03-09
1
-1
/
+1
|
*
oaipmh: HarvestPubmedWorker obsoleted by PubmedFTPWorker
Martin Czygan
2020-03-09
1
-34
/
+0
|
*
more pubmed adjustments
Martin Czygan
2020-02-22
2
-70
/
+118
|
*
pubmed ftp: fix url
Martin Czygan
2020-02-19
1
-4
/
+6
|
*
pubmed ftp harvest and KafkaBs4XmlPusher
Martin Czygan
2020-02-19
4
-1
/
+280
*
|
Merge branch 'bnewbold-elastic-v03b'
Bryan Newbold
2020-02-26
2
-46
/
+198
|
\
\
|
*
|
improve is_oa flag accuracy
Bryan Newbold
2020-02-26
1
-8
/
+4
|
*
|
ES container last tweaks
Bryan Newbold
2020-02-26
1
-0
/
+3
|
*
|
ES release: last minor tweaks
Bryan Newbold
2020-02-26
1
-2
/
+2
|
*
|
ES files: don't remove archive.org domains/hosts
Bryan Newbold
2020-02-07
1
-5
/
+0
|
*
|
ES releases: host/domain fixes
Bryan Newbold
2020-01-31
1
-2
/
+2
|
*
|
fix release es transform missing 'issue'
Bryan Newbold
2020-01-30
1
-0
/
+1
|
*
|
add upper-case work-around from kibana map join
Bryan Newbold
2020-01-30
1
-0
/
+1
|
*
|
tweak file ES archive.org domain tracking
Bryan Newbold
2020-01-30
1
-0
/
+6
|
*
|
implement host+domain parsing for file ES transform
Bryan Newbold
2020-01-30
1
-9
/
+5
|
*
|
fix ES file schema plural field names
Bryan Newbold
2020-01-29
1
-4
/
+3
|
*
|
elastic schema fixes
Bryan Newbold
2020-01-29
1
-0
/
+5
|
*
|
add country to v03b release schema
Bryan Newbold
2020-01-29
1
-0
/
+2
|
*
|
actually implement changelog transform
Bryan Newbold
2020-01-29
1
-17
/
+45
|
*
|
fix some transform bugs, add some tests
Bryan Newbold
2020-01-29
1
-6
/
+8
|
*
|
ES release schema updates
Bryan Newbold
2020-01-29
1
-5
/
+76
|
*
|
container ES schema changes
Bryan Newbold
2020-01-29
1
-16
/
+18
|
*
|
first implementation of ES file schema
Bryan Newbold
2020-01-29
2
-1
/
+46
*
|
|
Merge branch 'bnewbold-more-ingest' into 'master'
bnewbold
2020-02-25
1
-1
/
+37
|
\
\
\
|
*
|
|
entity worker: ingest more releases
Bryan Newbold
2020-02-22
1
-1
/
+37
*
|
|
|
add some more domain/rel URL mappings
Bryan Newbold
2020-02-22
1
-0
/
+9
|
/
/
/
*
|
|
Merge branch 'bnewbold-shadow-import'
Bryan Newbold
2020-02-19
2
-0
/
+196
|
\
\
\
|
*
|
|
remove arabesque short wayback URL hack
Bryan Newbold
2020-02-14
1
-6
/
+0
|
*
|
|
improve shadow import file url cleanup path
Bryan Newbold
2020-02-13
1
-2
/
+12
|
*
|
|
shadow import fixes from QA testing
Bryan Newbold
2020-02-13
1
-0
/
+6
|
*
|
|
shadow import: more filtering of file_meta fields
Bryan Newbold
2020-02-13
1
-0
/
+10
|
*
|
|
basic shadow importer
Bryan Newbold
2020-02-13
2
-0
/
+176
*
|
|
|
always crawl researchgate DOIs
Bryan Newbold
2020-02-18
1
-0
/
+2
*
|
|
|
ingest import: fix edit_extra path
Bryan Newbold
2020-02-18
1
-1
/
+1
*
|
|
|
ingest importer: edit_extra is a top-level key
Bryan Newbold
2020-02-18
1
-1
/
+1
*
|
|
|
ingest import: allow short version of corpus names
Bryan Newbold
2020-02-18
1
-0
/
+3
*
|
|
|
ingest importer: pass through link rel
Bryan Newbold
2020-02-18
1
-1
/
+6
|
|
_
|
/
|
/
|
|
*
|
|
harvest: log state on startup and use stderr for diagnostics
Martin Czygan
2020-02-14
3
-17
/
+22
|
/
/
*
|
add acceptlist override for biorxiv/medrxiv
Bryan Newbold
2020-02-10
1
-2
/
+12
*
|
check ingest_request_source existance for SPN as well as ingest
Bryan Newbold
2020-02-06
1
-0
/
+3
*
|
additional trusted link sources
Bryan Newbold
2020-02-06
1
-0
/
+3
*
|
add mag and s2 as trusted link sources
Bryan Newbold
2020-02-06
1
-1
/
+1
*
|
ingest worker: handle missing ingest_request_source
Bryan Newbold
2020-02-06
1
-0
/
+3
[prev]
[next]