aboutsummaryrefslogtreecommitdiffstats
path: root/pig/README.md
diff options
context:
space:
mode:
authorBryan Newbold <bnewbold@archive.org>2018-03-29 21:50:06 -0700
committerBryan Newbold <bnewbold@archive.org>2018-03-29 21:50:06 -0700
commit67e0a765749a4754ed353fe30c8e771d136322a4 (patch)
tree1e8d656ecc6f4830e5a3e787ba099f871a8137fa /pig/README.md
parentcb1582c44a000983a2150679c51b1baf22c09778 (diff)
downloadsandcrawler-67e0a765749a4754ed353fe30c8e771d136322a4.tar.gz
sandcrawler-67e0a765749a4754ed353fe30c8e771d136322a4.zip
import WIP on pig test setup
Diffstat (limited to 'pig/README.md')
-rw-r--r--pig/README.md28
1 files changed, 28 insertions, 0 deletions
diff --git a/pig/README.md b/pig/README.md
new file mode 100644
index 0000000..e47e31d
--- /dev/null
+++ b/pig/README.md
@@ -0,0 +1,28 @@
+
+As of March 2018, the archive runs Pig version 0.12.0, via CDH5.0.1 (Cloudera
+Distribution).
+
+## Development and Testing
+
+To run pig in development on your laptop, you can either use docker or
+
+https://hub.docker.com/r/chalimartines/local-pig
+
+ wget https://archive.cloudera.com/cdh5/cdh/5/pig-0.12.0-cdh5.0.1.tar.gz
+ tar xvf pig-*.tar.gz
+ ln -s pig-0.12.0-cdh5.0.1/pig-0.12.0-cdh5.0.1.jar pig-0.12.0-cdh5.0.1/pig.jar
+ ./pig-*/bin/pig -x local -version
+
+ #XXX:
+ #wget https://archive.cloudera.com/cdh5/cdh/5/hadoop-2.3.0-cdh5.0.1.tar.gz
+ #tar xvf hadoop-*.tar.gz
+ #export HADOOP_HOME=hadoop-2.3*
+
+Tests require python3, nosetests3, and pigpy. You can install these with:
+
+ pip install pipenv
+ pipenv install --three
+
+Then:
+
+ pipenv run nosetests3