diff options
| author | Bryan Newbold <bnewbold@archive.org> | 2018-05-21 18:35:30 -0700 |
|---|---|---|
| committer | Bryan Newbold <bnewbold@archive.org> | 2018-05-21 18:36:21 -0700 |
| commit | 6f5beb13c718acb28345bb037e3c24b3f366e170 (patch) | |
| tree | 28e4faa663951fd5ffd0510f94d32a1a361444e2 /scald-mvp | |
| parent | ed00bfffeac7b731f4edeaa4e283a3a09dcd2ee2 (diff) | |
| download | sandcrawler-6f5beb13c718acb28345bb037e3c24b3f366e170.tar.gz sandcrawler-6f5beb13c718acb28345bb037e3c24b3f366e170.zip | |
deps updates: cdh libs, hbase, custom spyglass
Diffstat (limited to 'scald-mvp')
| -rw-r--r-- | scald-mvp/README.md | 3 | ||||
| -rw-r--r-- | scald-mvp/build.sbt | 7 |
2 files changed, 7 insertions, 3 deletions
diff --git a/scald-mvp/README.md b/scald-mvp/README.md index 31c174b..7afcef8 100644 --- a/scald-mvp/README.md +++ b/scald-mvp/README.md @@ -45,3 +45,6 @@ in SpyGlass repo: # then build here sbt assembly + +The medium-term plan here is to push the custom SpyGlass jar as a static maven +repo to an archive.org item, and point build.sbt to that folder. diff --git a/scald-mvp/build.sbt b/scald-mvp/build.sbt index 213c78b..5030eba 100644 --- a/scald-mvp/build.sbt +++ b/scald-mvp/build.sbt @@ -1,7 +1,7 @@ import Dependencies._ -val hadoopVersion = "2.5.0" // IA cluster 2018-05-21: 2.5.0-cdh5.3.1 -val hbaseVersion = "0.98.6" // IA cluster 2018-05-21: 0.98.6-cdh5.3.1 +val hadoopVersion = "2.5.0-cdh5.3.1" // IA cluster 2018-05-21: 2.5.0-cdh5.3.1 +val hbaseVersion = "0.98.6-cdh5.3.1" // IA cluster 2018-05-21: 0.98.6-cdh5.3.1 lazy val root = (project in file(".")). @@ -25,7 +25,8 @@ lazy val root = (project in file(".")). libraryDependencies += "org.apache.hadoop" % "hadoop-common" % hadoopVersion, libraryDependencies += "org.apache.hadoop" % "hadoop-client" % hadoopVersion, libraryDependencies += "org.apache.hadoop" % "hadoop-mapreduce-client-jobclient" % hadoopVersion classifier "tests", - libraryDependencies += "parallelai" % "parallelai.spyglass" % "2.11_0.17.2_5.3.0", + libraryDependencies += "org.apache.hbase" % "hbase-common" % hbaseVersion, + libraryDependencies += "parallelai" % "parallelai.spyglass" % "2.11_0.17.2_cdh5.3.1", // cargo-culted from twitter/scalding's build.sbt // hint via https://stackoverflow.com/questions/23280494/sbt-assembly-error-deduplicate-different-file-contents-found-in-the-following#23280952 |