diff options
| author | Bryan Newbold <bnewbold@archive.org> | 2018-05-01 22:33:52 +0000 |
|---|---|---|
| committer | Bryan Newbold <bnewbold@archive.org> | 2018-07-06 14:48:57 +0000 |
| commit | fde027fb269214a7e8f71f5717bcf569014b6661 (patch) | |
| tree | 1cbbe97e681d3e0420b00a967b1cd89e380992c2 /pig | |
| parent | 41c26d2b9ac215f2dacd15f6b7433b909cafb552 (diff) | |
| download | sandcrawler-fde027fb269214a7e8f71f5717bcf569014b6661.tar.gz sandcrawler-fde027fb269214a7e8f71f5717bcf569014b6661.zip | |
possibly-broken version of hbase-count-rows.pig
This just worked a minute ago, but now throws:
org.apache.hadoop.hbase.DoNotRetryIOException: java.lang.NoClassDefFoundError: org/apache/hadoop/hbase/util/ByteStringer
Diffstat (limited to 'pig')
| -rw-r--r-- | pig/hbase-count-rows.pig | 13 |
1 files changed, 13 insertions, 0 deletions
diff --git a/pig/hbase-count-rows.pig b/pig/hbase-count-rows.pig new file mode 100644 index 0000000..523beb2 --- /dev/null +++ b/pig/hbase-count-rows.pig @@ -0,0 +1,13 @@ + +REGISTER /usr/lib/hbase/hbase-client-0.98.6-cdh5.3.1.jar +REGISTER /usr/lib/hbase/hbase-common-0.98.6-cdh5.3.1.jar + +set hbase.zookeeper.quorum 'mtrcs-zk1.us.archive.org,mtrcs-zk2.us.archive.org,mtrcs-zk3.us.archive.org' + +data = LOAD 'hbase://wbgrp-journal-extract-0-qa' + USING org.apache.pig.backend.hadoop.hbase.HBaseStorage('grobid0:status_code', '-loadKey true') + AS (key:CHARARRAY, status:CHARARRAY); + +data_group = GROUP data ALL; +data_count = FOREACH data_group GENERATE COUNT(data); +DUMP data_count; |