diff options
| author | Bryan Newbold <bnewbold@archive.org> | 2020-01-10 16:18:38 -0800 |
|---|---|---|
| committer | Bryan Newbold <bnewbold@archive.org> | 2020-01-10 16:18:38 -0800 |
| commit | 566469783d6ea0e578d314232bff7d622c671241 (patch) | |
| tree | 8b0a98a169a9439a67120f6f4fc8bf495bf08354 | |
| parent | 867d71bbfd89be44fedf5955a60cb1e8b774b390 (diff) | |
| download | sandcrawler-566469783d6ea0e578d314232bff7d622c671241.tar.gz sandcrawler-566469783d6ea0e578d314232bff7d622c671241.zip | |
setup notes and CI (for psycopg2 dependency)
| -rw-r--r-- | .gitlab-ci.yml | 2 | ||||
| -rw-r--r-- | README.md | 5 |
2 files changed, 6 insertions, 1 deletions
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 5053a57..6a948dd 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -5,7 +5,7 @@ before_script: - echo "deb https://dl.bintray.com/sbt/debian /" | tee -a /etc/apt/sources.list.d/sbt.list - apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 2EE0EA64E40A89B84B2DF73499E82A75642AC823 - apt update -qy - - apt install -y python3-dev python3-pip python3-wheel libjpeg-dev openjdk-8-jdk-headless sbt + - apt install -y python3-dev python3-pip python3-wheel libjpeg-dev openjdk-8-jdk-headless sbt libpq-dev python-dev - pip3 install pipenv - pipenv --version @@ -38,3 +38,8 @@ The `./please` python3 wrapper script is a helper for running jobs (python or scalding) on the IA Hadoop cluster. You'll need to run the setup/dependency tasks first; see README files in subdirectories. +## Running Python Code + +You need python3.5 (or python3.6+ and `pyenv`) and `pipenv` to set up the +environment. You may also need the debian packages `libpq-dev` and ` +python-dev` to install some dependencies. |