diff options
| author | Ellen Spertus <ellen.spertus@gmail.com> | 2018-05-31 19:12:49 +0000 |
|---|---|---|
| committer | Ellen Spertus <ellen.spertus@gmail.com> | 2018-05-31 19:12:49 +0000 |
| commit | 076d44fa40ce93cb6cc14fa600f91f4c9ef5d1e1 (patch) | |
| tree | fbc13915539137026a56d35ffe76ffdf51fb9651 /scalding | |
| parent | fb54ee7fcf58962473011237e59e08c0eec00e4a (diff) | |
| download | sandcrawler-076d44fa40ce93cb6cc14fa600f91f4c9ef5d1e1.tar.gz sandcrawler-076d44fa40ce93cb6cc14fa600f91f4c9ef5d1e1.zip | |
Added tip on OutOfMemoryError.
Diffstat (limited to 'scalding')
| -rw-r--r-- | scalding/README.md | 6 |
1 files changed, 5 insertions, 1 deletions
diff --git a/scalding/README.md b/scalding/README.md index 9df06a6..13c2e1a 100644 --- a/scalding/README.md +++ b/scalding/README.md @@ -1,4 +1,3 @@ - This directory contains Hadoop map/reduce jobs written in Scala (compiled to the JVM) using the Scalding framework. @@ -27,6 +26,11 @@ Run on cluster: com.twitter.scalding.Tool sandcrawler.HBaseRowCountJob --hdfs \ --app.conf.path thing.conf \ --output hdfs:///user/bnewbold/spyglass_out_test + +If your `sbt` task fails with this error: + + java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: Metaspace +try restarting `sbt` with more memory (e.g., `sbt -mem 2048`). ## Building SpyGlass Jar |