sandcrawler/mapreduce, branch bnewbold-args [no description] https://git.bnewbold.net/sandcrawler/atom?h=bnewbold-args 2018-06-04T19:42:29+00:00 bnewbold-dev > wbgrp-svc263 2018-06-04T19:42:29+00:00 Bryan Newbold bnewbold@archive.org 2018-05-31T02:40:05+00:00 urn:sha1:d2dd016aa8da93ad14654237dbb7cfac214f9da8 This is a new production VM running an HBase-Thrift gateway actually fix oversize inserts 2018-05-08T03:24:01+00:00 Bryan Newbold bnewbold@archive.org 2018-05-08T03:24:01+00:00 urn:sha1:0c398392aa298d28694bf5bd37d3e4912de8a2f5 XML size limit 2018-04-26T18:24:42+00:00 Bryan Newbold bnewbold@archive.org 2018-04-26T18:24:42+00:00 urn:sha1:ee6ce29e7987f936536a0ef128d3a96cc1df3d86 force_existing flag for extraction 2018-04-19T05:15:02+00:00 Bryan Newbold bnewbold@archive.org 2018-04-19T05:14:33+00:00 urn:sha1:df23b6f45922875f0bf657aea3b8c3fb4451469d NLineInputFormat requires RawProtocol 2018-04-19T05:15:02+00:00 Bryan Newbold bnewbold@archive.org 2018-04-15T05:44:59+00:00 urn:sha1:e0d1e381bf536d1c077546526c21eab909444193 Should make this a command line argument or something. Want one in HADOOP, the other for local/tests/inline/etc. local mrjob config 2018-04-19T05:15:02+00:00 Bryan Newbold bnewbold@archive.org 2018-04-15T05:44:07+00:00 urn:sha1:a8a568f03d7f537a8683adf23f6643c7704e8d3d switch to new (local) sentry instance 2018-04-19T05:13:59+00:00 Bryan Newbold bnewbold@archive.org 2018-04-19T05:13:59+00:00 urn:sha1:c3e925cdd00c28531ee37ac7ada5cdee229762db update Pipfile.lock (new pluggy) 2018-04-16T19:47:13+00:00 Bryan Newbold bnewbold@archive.org 2018-04-16T19:46:54+00:00 urn:sha1:e70ac19416d7f2d8f91da3e700e0c3b88f3965bc use NLineInputFormat so we can control split size 2018-04-11T05:33:53+00:00 Bryan Newbold bnewbold@archive.org 2018-04-11T05:33:53+00:00 urn:sha1:43f4a6ec3895f4ac5a7db0dfa237aed44f52358b revert PYTHONPATH in cmdenv 2018-04-11T05:19:34+00:00 Bryan Newbold bnewbold@archive.org 2018-04-11T05:19:32+00:00 urn:sha1:be1704a419a1e916bb0055e2b40d2db026976001 Seemed to break hadoop jobs for some reason