aboutsummaryrefslogtreecommitdiffstats
path: root/python
diff options
context:
space:
mode:
authorBryan Newbold <bnewbold@archive.org>2020-01-02 18:10:41 -0800
committerBryan Newbold <bnewbold@archive.org>2020-01-02 18:13:03 -0800
commit34f8d3d32845774d7fc8690350084eabff54e44f (patch)
treef016f7449e8eef538a4ba367fe0f5481742e2cb8 /python
parentddf56878e867e3556cb9c6cb6df6e60cff4e21fa (diff)
downloadsandcrawler-34f8d3d32845774d7fc8690350084eabff54e44f.tar.gz
sandcrawler-34f8d3d32845774d7fc8690350084eabff54e44f.zip
TODO: check out python ratelimit library
Diffstat (limited to 'python')
-rw-r--r--python/TODO2
1 files changed, 1 insertions, 1 deletions
diff --git a/python/TODO b/python/TODO
index 89cec83..58a463f 100644
--- a/python/TODO
+++ b/python/TODO
@@ -4,4 +4,4 @@ ingest crawler:
- remove most SPNv1/v2 path selection
- landing page + fulltext hops only (short recursion depth)
- use wayback client library instead of requests to fetch content
-
+- https://pypi.org/project/ratelimit/