From e433990172c157707d92452652aefe2f21b6a4a0 Mon Sep 17 00:00:00 2001 From: Bryan Newbold Date: Mon, 2 Jan 2023 19:10:01 -0800 Subject: python-specific README file --- python/TODO | 7 ------- 1 file changed, 7 deletions(-) delete mode 100644 python/TODO (limited to 'python/TODO') diff --git a/python/TODO b/python/TODO deleted file mode 100644 index 58a463f..0000000 --- a/python/TODO +++ /dev/null @@ -1,7 +0,0 @@ - -ingest crawler: -- SPNv2 only - - remove most SPNv1/v2 path selection -- landing page + fulltext hops only (short recursion depth) -- use wayback client library instead of requests to fetch content -- https://pypi.org/project/ratelimit/ -- cgit v1.2.3