aboutsummaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
authorBryan Newbold <bnewbold@archive.org>2018-05-21 19:19:29 -0700
committerBryan Newbold <bnewbold@archive.org>2018-05-21 19:19:29 -0700
commit776ce9a17547e07baabf7c4e9fa572d4d9d4eeb7 (patch)
tree9f0d8f8c71bfb4599496c0006bdb64ed5dc37f92
parent3b64d9c4f852b29964eca0e215d636bc37f0d856 (diff)
downloadsandcrawler-776ce9a17547e07baabf7c4e9fa572d4d9d4eeb7.tar.gz
sandcrawler-776ce9a17547e07baabf7c4e9fa572d4d9d4eeb7.zip
update README with invocations
-rw-r--r--scald-mvp/README.md13
1 files changed, 13 insertions, 0 deletions
diff --git a/scald-mvp/README.md b/scald-mvp/README.md
index 7afcef8..e41e9ec 100644
--- a/scald-mvp/README.md
+++ b/scald-mvp/README.md
@@ -31,6 +31,19 @@ helpful for debugging dependency woes:
sbt dependencyTree
+testing the spyglass example program (expect a table error):
+
+ hadoop jar scald-mvp-assembly-0.1.0-SNAPSHOT.jar com.twitter.scalding.Tool example.SimpleHBaseSourceExample --hdfs --output hdfs:///user/bnewbold/spyglass_out_test --app.conf.path thing.conf --debug true
+ # org.apache.hadoop.hbase.TableNotFoundException: table_name
+
+running a spyglass job (gives a nullpointer exception):
+
+ hadoop jar scald-mvp-assembly-0.1.0-SNAPSHOT.jar com.twitter.scalding.Tool sandcrawler.HBaseRowCountJob --hdfs --output hdfs:///user/bnewbold/spyglass_out_test --app.conf.path thing.conf
+
+ # Caused by: java.lang.NullPointerException
+ # at parallelai.spyglass.hbase.HBaseSource.<init>(HBaseSource.scala:48)
+ # at sandcrawler.HBaseRowCountJob.<init>(HBaseRowCountJob.scala:17)
+
## Custom build
in SpyGlass repo: