diff options
| author | Bryan Newbold <bnewbold@archive.org> | 2023-01-02 19:10:01 -0800 |
|---|---|---|
| committer | Bryan Newbold <bnewbold@archive.org> | 2023-01-02 19:10:01 -0800 |
| commit | e433990172c157707d92452652aefe2f21b6a4a0 (patch) | |
| tree | de662f71c65447017828de4f10fe43eb5705c40f /python/TODO | |
| parent | b7e4629f3c84f35af5ad62346a9480bea957c719 (diff) | |
| download | sandcrawler-e433990172c157707d92452652aefe2f21b6a4a0.tar.gz sandcrawler-e433990172c157707d92452652aefe2f21b6a4a0.zip | |
python-specific README file
Diffstat (limited to 'python/TODO')
| -rw-r--r-- | python/TODO | 7 |
1 files changed, 0 insertions, 7 deletions
diff --git a/python/TODO b/python/TODO deleted file mode 100644 index 58a463f..0000000 --- a/python/TODO +++ /dev/null @@ -1,7 +0,0 @@ - -ingest crawler: -- SPNv2 only - - remove most SPNv1/v2 path selection -- landing page + fulltext hops only (short recursion depth) -- use wayback client library instead of requests to fetch content -- https://pypi.org/project/ratelimit/ |