sandcrawler/pig, branch bnewbold-args [no description] https://git.bnewbold.net/sandcrawler/atom?h=bnewbold-args 2018-05-08T17:06:20+00:00 fix tests post-DISTINCT 2018-05-08T17:06:20+00:00 Bryan Newbold bnewbold@archive.org 2018-05-08T17:06:14+00:00 urn:sha1:18a55d37a87d4391bd8161201c523dd7d7f0f1e7 Confirms it's working! distinct on SHA1 in cdx scripts 2018-05-08T16:58:24+00:00 Bryan Newbold bnewbold@archive.org 2018-05-08T16:58:24+00:00 urn:sha1:1831a3b4495aee275e4b4b187fa545eba75eb87b pig cdx join improvements 2018-05-08T16:58:09+00:00 Bryan Newbold bnewbold@archive.org 2018-05-08T16:58:09+00:00 urn:sha1:64503ce8fb755384623821bfabfa81bbb37d8f6e how to run pig in production 2018-05-08T06:44:39+00:00 Bryan Newbold bnewbold@archive.org 2018-05-08T06:44:39+00:00 urn:sha1:681b085bc2a090b8db366c54780f1ec81d811403 WIP on filter-cdx-join-urls.pig 2018-05-08T06:41:10+00:00 Bryan Newbold bnewbold@archive.org 2018-05-08T06:41:10+00:00 urn:sha1:2a1c887309305187d785b34a16c1868d26cb3273 pig script to filter GWB CDX by SURT regexes 2018-05-08T05:11:18+00:00 Bryan Newbold bnewbold@archive.org 2018-05-08T05:10:51+00:00 urn:sha1:d1401444dbfb515e62094f873d520a23ccbc29d9 improve pig helper 2018-05-08T05:11:18+00:00 Bryan Newbold bnewbold@archive.org 2018-05-08T05:10:18+00:00 urn:sha1:81d2f6290fff487f0f49b109227443c0f8a7aedb try pig env again 2018-04-06T22:27:27+00:00 Bryan Newbold bnewbold@archive.org 2018-04-06T22:27:27+00:00 urn:sha1:401b09afb5665356a44291af5b42a3cd9836c0fb use IA mirror for pig download 2018-04-06T22:17:32+00:00 Bryan Newbold bnewbold@archive.org 2018-04-06T22:17:32+00:00 urn:sha1:b5b266dc45d64e94bb9d721643d06965b7872963 shift docs around a bit 2018-04-03T02:25:57+00:00 Bryan Newbold bnewbold@archive.org 2018-04-03T02:25:57+00:00 urn:sha1:9d4520e8e18d7bf9b36d98d330417360194e80a3