diff options
| author | Bryan Newbold <bnewbold@archive.org> | 2018-05-07 22:10:51 -0700 |
|---|---|---|
| committer | Bryan Newbold <bnewbold@archive.org> | 2018-05-07 22:11:18 -0700 |
| commit | d1401444dbfb515e62094f873d520a23ccbc29d9 (patch) | |
| tree | 418a21b93261230b006127107b124e5c12236ab7 /pig/tests/files/papers_edu_tilde.cdx | |
| parent | 81d2f6290fff487f0f49b109227443c0f8a7aedb (diff) | |
| download | sandcrawler-d1401444dbfb515e62094f873d520a23ccbc29d9.tar.gz sandcrawler-d1401444dbfb515e62094f873d520a23ccbc29d9.zip | |
pig script to filter GWB CDX by SURT regexes
Diffstat (limited to 'pig/tests/files/papers_edu_tilde.cdx')
| -rw-r--r-- | pig/tests/files/papers_edu_tilde.cdx | 15 |
1 files changed, 15 insertions, 0 deletions
diff --git a/pig/tests/files/papers_edu_tilde.cdx b/pig/tests/files/papers_edu_tilde.cdx new file mode 100644 index 0000000..47ca069 --- /dev/null +++ b/pig/tests/files/papers_edu_tilde.cdx @@ -0,0 +1,15 @@ +#http://www.stanford.edu:80/~johntayl/Papers/taylor2.pdf +#http://met.nps.edu/~mtmontgo/papers/isabel_part2.pdf +#http://www.pitt.edu:80/~druzdzel/psfiles/ecai06.pdf +#http://www.comp.hkbu.edu.hk/~ymc/papers/conference/ijcnn03_710.pdf + +# should be 6 matches: +hk,edu,hkbu,comp)/~ymc/papers/conference/ijcnn03_710.pdf 20170706005950 http://mit.edu/file.pdf application/pdf 200 MQHD36X5MNZPWFNMD5LFOYZSFGCHUN3V - - 123 456 CRAWL/CRAWL.warc.gz +edu,stanford,www)/~johntayl/Papers/taylor2.pdf 20170706005950 http://mit.edu/file.pdf application/pdf 200 MQHD36X5MNZPWFNMD5LFOYZSFGCHUN3V - - 123 456 CRAWL/CRAWL.warc.gz +edu,nps,met)/~mtmontgo/papers/isabel_part2.pdf 20170706005950 http://mit.edu/file.pdf application/pdf 200 MQHD36X5MNZPWFNMD5LFOYZSFGCHUN3V - - 123 456 CRAWL/CRAWL.warc.gz +edu,pitt,www)/~druzdzel/psfiles/ecai06.pdf 20170706005950 http://mit.edu/file.pdf application/pdf 200 MQHD36X5MNZPWFNMD5LFOYZSFGCHUN3V - - 123 456 CRAWL/CRAWL.warc.gz +jp,ac,pitt,www)/~druzdzel/psfiles/ecai06.pdf 20170706005950 http://mit.edu/file.pdf application/pdf 200 MQHD36X5MNZPWFNMD5LFOYZSFGCHUN3V - - 123 456 CRAWL/CRAWL.warc.gz +co,edu,pitt,www)/~druzdzel/psfiles/ecai06.pdf 20170706005950 http://mit.edu/file.pdf application/pdf 200 MQHD36X5MNZPWFNMD5LFOYZSFGCHUN3V - - 123 456 CRAWL/CRAWL.warc.gz + +# NOT: +com,corp,edu,pitt,www)/~druzdzel/psfiles/ecai06.pdf 20170706005950 http://mit.edu/file.pdf application/pdf 200 MQHD36X5MNZPWFNMD5LFOYZSFGCHUN3V - - 123 456 CRAWL/CRAWL.warc.gz |