aboutsummaryrefslogtreecommitdiffstats
path: root/proposals
diff options
context:
space:
mode:
authorBryan Newbold <bnewbold@archive.org>2021-10-15 18:17:32 -0700
committerBryan Newbold <bnewbold@archive.org>2021-10-15 18:17:32 -0700
commit12d041b781912dc376444198c920ade2d6cee7c8 (patch)
treeb840986d37cfa37e2eb761bdfe426294ccc59da1 /proposals
parent8310ac89e08afc322122ba9c9365d32950b062d7 (diff)
downloadsandcrawler-12d041b781912dc376444198c920ade2d6cee7c8.tar.gz
sandcrawler-12d041b781912dc376444198c920ade2d6cee7c8.zip
commit SPN account changes
Diffstat (limited to 'proposals')
-rw-r--r--proposals/2021-09-21_spn_accounts.md14
1 files changed, 14 insertions, 0 deletions
diff --git a/proposals/2021-09-21_spn_accounts.md b/proposals/2021-09-21_spn_accounts.md
new file mode 100644
index 0000000..e41c162
--- /dev/null
+++ b/proposals/2021-09-21_spn_accounts.md
@@ -0,0 +1,14 @@
+
+Formalization of SPNv2 API requests from fatcat/sandcrawler
+
+Create two new system accounts, one for regular/daily ingest requests, one for
+priority requests (save-paper-now or as a flag with things like fatcat-ingest;
+"interactive"). These accounts should have @archive.org emails. Request the
+daily one to have the current rate limit as bnewbold@archive.org account; the
+priority queue can have less.
+
+Create new ingest kafka queues from scratch, one for priority and one for
+regular. Chose sizes carefully, probably keep 24x for the regular and do 6x or
+so (small) for priority queue.
+
+Deploy new priority workers; reconfigure/deploy broadly.