aboutsummaryrefslogtreecommitdiffstats
path: root/scald-mvp/README.md
diff options
context:
space:
mode:
authorBryan Newbold <bnewbold@archive.org>2018-05-21 18:34:51 -0700
committerBryan Newbold <bnewbold@archive.org>2018-05-21 18:34:51 -0700
commited00bfffeac7b731f4edeaa4e283a3a09dcd2ee2 (patch)
tree25d6a284838ba017d34c901f715a672796a8974a /scald-mvp/README.md
parent465077725b238ecf4f1355e60516818aa3117fc3 (diff)
downloadsandcrawler-ed00bfffeac7b731f4edeaa4e283a3a09dcd2ee2.tar.gz
sandcrawler-ed00bfffeac7b731f4edeaa4e283a3a09dcd2ee2.zip
docs of how to munge around custom spyglass jars
Diffstat (limited to 'scald-mvp/README.md')
-rw-r--r--scald-mvp/README.md19
1 files changed, 19 insertions, 0 deletions
diff --git a/scald-mvp/README.md b/scald-mvp/README.md
index 6c7ec76..31c174b 100644
--- a/scald-mvp/README.md
+++ b/scald-mvp/README.md
@@ -26,3 +26,22 @@ running on my laptop:
later, using hadop command instead:
hadoop jar scald-mvp-assembly-0.1.0-SNAPSHOT.jar com.twitter.scalding.Tool example.WordCountJob --hdfs --input hdfs:///user/bnewbold/dummy.txt --output hdfs:///user/bnewbold/test_scalding_out3
+
+helpful for debugging dependency woes:
+
+ sbt dependencyTree
+
+## Custom build
+
+in SpyGlass repo:
+
+ # This builds the new .jar and installs it in the (laptop local) ~/.m2
+ # repository
+ mvn clean install -U
+
+ # Copy that .jar (and associated pom.xml) over to where sbt can find it
+ mkdir -p ~/.sbt/preloaded/parallelai/
+ cp -r ~/.m2/repository/parallelai/parallelai.spyglass ~/.sbt/preloaded/parallelai/
+
+ # then build here
+ sbt assembly