diff options
| author | Bryan Newbold <bnewbold@archive.org> | 2018-03-29 21:50:06 -0700 |
|---|---|---|
| committer | Bryan Newbold <bnewbold@archive.org> | 2018-03-29 21:50:06 -0700 |
| commit | 67e0a765749a4754ed353fe30c8e771d136322a4 (patch) | |
| tree | 1e8d656ecc6f4830e5a3e787ba099f871a8137fa /pig/README.md | |
| parent | cb1582c44a000983a2150679c51b1baf22c09778 (diff) | |
| download | sandcrawler-67e0a765749a4754ed353fe30c8e771d136322a4.tar.gz sandcrawler-67e0a765749a4754ed353fe30c8e771d136322a4.zip | |
import WIP on pig test setup
Diffstat (limited to 'pig/README.md')
| -rw-r--r-- | pig/README.md | 28 |
1 files changed, 28 insertions, 0 deletions
diff --git a/pig/README.md b/pig/README.md new file mode 100644 index 0000000..e47e31d --- /dev/null +++ b/pig/README.md @@ -0,0 +1,28 @@ + +As of March 2018, the archive runs Pig version 0.12.0, via CDH5.0.1 (Cloudera +Distribution). + +## Development and Testing + +To run pig in development on your laptop, you can either use docker or + +https://hub.docker.com/r/chalimartines/local-pig + + wget https://archive.cloudera.com/cdh5/cdh/5/pig-0.12.0-cdh5.0.1.tar.gz + tar xvf pig-*.tar.gz + ln -s pig-0.12.0-cdh5.0.1/pig-0.12.0-cdh5.0.1.jar pig-0.12.0-cdh5.0.1/pig.jar + ./pig-*/bin/pig -x local -version + + #XXX: + #wget https://archive.cloudera.com/cdh5/cdh/5/hadoop-2.3.0-cdh5.0.1.tar.gz + #tar xvf hadoop-*.tar.gz + #export HADOOP_HOME=hadoop-2.3* + +Tests require python3, nosetests3, and pigpy. You can install these with: + + pip install pipenv + pipenv install --three + +Then: + + pipenv run nosetests3 |