diff options
| author | Bryan Newbold <bnewbold@archive.org> | 2020-01-08 13:19:26 -0800 |
|---|---|---|
| committer | Bryan Newbold <bnewbold@archive.org> | 2020-01-08 13:19:29 -0800 |
| commit | 2035b62d6e46c1c57243ee3e68d1067a30791f54 (patch) | |
| tree | 1d2632881b0ad4830594490ea8e2943b8e204494 /python_hadoop/mrjob.conf | |
| parent | 1ca8b792709dde71f350827fdef6e6596dda55a0 (diff) | |
| download | sandcrawler-2035b62d6e46c1c57243ee3e68d1067a30791f54.tar.gz sandcrawler-2035b62d6e46c1c57243ee3e68d1067a30791f54.zip | |
refactor CdxApiClient, add tests
- always use auth token and get full CDX rows
- simplify to "fetch" (exact url/dt match) and "lookup_best" methods
- all redirect stuff will be moved to a higher level
Diffstat (limited to 'python_hadoop/mrjob.conf')
0 files changed, 0 insertions, 0 deletions