aboutsummaryrefslogtreecommitdiffstats
path: root/tests/test_grobid_parse.py
diff options
context:
space:
mode:
authorBryan Newbold <bnewbold@archive.org>2022-01-11 12:54:59 -0800
committerBryan Newbold <bnewbold@archive.org>2022-01-11 12:54:59 -0800
commit52576d07393b3511474ef077171c012244b3ccd9 (patch)
tree481102a25974118b43f5693da975da1cff8ef614 /tests/test_grobid_parse.py
parent3d830aa01867e1324d1c85ba681cf799d98605bc (diff)
downloadfatcat-scholar-52576d07393b3511474ef077171c012244b3ccd9.tar.gz
fatcat-scholar-52576d07393b3511474ef077171c012244b3ccd9.zip
robots.txt: block /search for large crawlers
Diffstat (limited to 'tests/test_grobid_parse.py')
0 files changed, 0 insertions, 0 deletions