diff options
| author | Bryan Newbold <bnewbold@archive.org> | 2020-05-21 19:51:23 -0700 |
|---|---|---|
| committer | Bryan Newbold <bnewbold@archive.org> | 2020-05-21 19:51:23 -0700 |
| commit | 55bb09006e984da217cbfdf4145aaad16b0b5cac (patch) | |
| tree | 73084a62056ed25424585348dced6b52c74ec296 | |
| parent | 0063be61247aa4ea697568ce9741f6857b80d595 (diff) | |
| download | fatcat-scholar-55bb09006e984da217cbfdf4145aaad16b0b5cac.tar.gz fatcat-scholar-55bb09006e984da217cbfdf4145aaad16b0b5cac.zip | |
helpers to fetch small-ish data samples
| -rw-r--r-- | Makefile | 8 |
1 files changed, 8 insertions, 0 deletions
@@ -21,6 +21,14 @@ dev: ## Run web service locally, with reloading run: ## Run web service under gunicorn pipenv run gunicorn fatcat_scholar.web:app -w 4 -k uvicorn.workers.UvicornWorker +.PHONY: fetch-works +fetch-works: ## Fetches some works from any release .json in the data dir + cat data/release_*.json | jq . -c | pipenv run python -m fatcat_scholar.work_pipeline run_releases | pv -l > data/work_intermediate.json + +.PHONY: fetch-sim +fetch-sim: ## Fetches some SIM pages + pipenv run python -m fatcat_scholar.sim_pipeline run_issue_db --limit 500 | pv -l > data/sim_intermediate.json + .PHONY: dev-index dev-index: ## Delete/Create DEV elasticsearch fulltext index locally http delete ":9200/dev_scholar_fulltext_v01" && true |