diff options
Diffstat (limited to 'python/TODO')
| -rw-r--r-- | python/TODO | 7 |
1 files changed, 7 insertions, 0 deletions
diff --git a/python/TODO b/python/TODO new file mode 100644 index 0000000..58a463f --- /dev/null +++ b/python/TODO @@ -0,0 +1,7 @@ + +ingest crawler: +- SPNv2 only + - remove most SPNv1/v2 path selection +- landing page + fulltext hops only (short recursion depth) +- use wayback client library instead of requests to fetch content +- https://pypi.org/project/ratelimit/ |