diff options
| author | Bryan Newbold <bnewbold@archive.org> | 2022-01-11 12:54:59 -0800 |
|---|---|---|
| committer | Bryan Newbold <bnewbold@archive.org> | 2022-01-11 12:54:59 -0800 |
| commit | 52576d07393b3511474ef077171c012244b3ccd9 (patch) | |
| tree | 481102a25974118b43f5693da975da1cff8ef614 /fatcat_scholar/transform.py | |
| parent | 3d830aa01867e1324d1c85ba681cf799d98605bc (diff) | |
| download | fatcat-scholar-52576d07393b3511474ef077171c012244b3ccd9.tar.gz fatcat-scholar-52576d07393b3511474ef077171c012244b3ccd9.zip | |
robots.txt: block /search for large crawlers
Diffstat (limited to 'fatcat_scholar/transform.py')
0 files changed, 0 insertions, 0 deletions