sandcrawler/mapreduce, branch master [no description] https://git.bnewbold.net/sandcrawler/atom?h=master 2018-08-24T19:28:51+00:00 rename ./mapreduce to ./python 2018-08-24T19:28:51+00:00 Bryan Newbold bnewbold@archive.org 2018-08-24T19:28:51+00:00 urn:sha1:3782311e29b7e477e1936c89f55ff6483fd02e65 extraction: do want content, not text 2018-08-21T23:28:55+00:00 Bryan Newbold bnewbold@archive.org 2018-08-21T23:28:53+00:00 urn:sha1:6c92ee4c0b137c28abd03ed72190210da8a1e72b XML can have non-unicode characters? Who knew. extraction: status reporting tweaks 2018-08-21T17:36:09+00:00 Bryan Newbold bnewbold@archive.org 2018-08-21T17:36:07+00:00 urn:sha1:139ca7e5a90d49c33e23de781b7e4ac21e868fac Improvements to how the extraction function in the extraction script reports status (in output, hbase, and counters) monkey-patch SHA-1 blacklist 2018-07-05T20:35:07+00:00 Bryan Newbold bnewbold@archive.org 2018-07-05T20:35:07+00:00 urn:sha1:41c26d2b9ac215f2dacd15f6b7433b909cafb552 doc improvements and fixes to 'please' helper 2018-06-15T00:41:33+00:00 Bryan Newbold bnewbold@archive.org 2018-06-15T00:41:33+00:00 urn:sha1:c23ccd1f2d03ad65ee83b8eca8c407d12ecd54e1 bnewbold-dev > wbgrp-svc263 2018-06-04T19:42:29+00:00 Bryan Newbold bnewbold@archive.org 2018-05-31T02:40:05+00:00 urn:sha1:d2dd016aa8da93ad14654237dbb7cfac214f9da8 This is a new production VM running an HBase-Thrift gateway actually fix oversize inserts 2018-05-08T03:24:01+00:00 Bryan Newbold bnewbold@archive.org 2018-05-08T03:24:01+00:00 urn:sha1:0c398392aa298d28694bf5bd37d3e4912de8a2f5 XML size limit 2018-04-26T18:24:42+00:00 Bryan Newbold bnewbold@archive.org 2018-04-26T18:24:42+00:00 urn:sha1:ee6ce29e7987f936536a0ef128d3a96cc1df3d86 force_existing flag for extraction 2018-04-19T05:15:02+00:00 Bryan Newbold bnewbold@archive.org 2018-04-19T05:14:33+00:00 urn:sha1:df23b6f45922875f0bf657aea3b8c3fb4451469d NLineInputFormat requires RawProtocol 2018-04-19T05:15:02+00:00 Bryan Newbold bnewbold@archive.org 2018-04-15T05:44:59+00:00 urn:sha1:e0d1e381bf536d1c077546526c21eab909444193 Should make this a command line argument or something. Want one in HADOOP, the other for local/tests/inline/etc.