From 905c116821dbcf0103323fcf8f0b58d2dfa81ddf Mon Sep 17 00:00:00 2001
From: Bryan Newbold <bnewbold@archive.org>
Date: Thu, 26 Dec 2019 21:35:18 -0800
Subject: update TODO

---
 python/TODO | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

(limited to 'python')

diff --git a/python/TODO b/python/TODO
index 6b05646..89cec83 100644
--- a/python/TODO
+++ b/python/TODO
@@ -1 +1,7 @@
-- refactor extractor common code into a shared file
+
+ingest crawler:
+- SPNv2 only
+    - remove most SPNv1/v2 path selection
+- landing page + fulltext hops only (short recursion depth)
+- use wayback client library instead of requests to fetch content
+
-- 
cgit v1.2.3