diff options
| author | Bryan Newbold <bnewbold@archive.org> | 2021-09-30 15:24:22 -0700 |
|---|---|---|
| committer | Bryan Newbold <bnewbold@archive.org> | 2021-09-30 15:24:24 -0700 |
| commit | e4800fc4d0d0467d0e34a4059b941d001916e232 (patch) | |
| tree | e789c6bfe7dfa95bc497b0329c9f9939864b1b71 /notes/url_pattern_heuristic_backfill.txt | |
| parent | 1c43b0d2a663815c7cb43c918933588f5184c714 (diff) | |
| download | sandcrawler-e4800fc4d0d0467d0e34a4059b941d001916e232.tar.gz sandcrawler-e4800fc4d0d0467d0e34a4059b941d001916e232.zip | |
new 'daily' and 'priority' ingest request topics
The old ingest request queue was always getting lopsided, suspect
because it was scaled up (additional partitions) at some point in the
past, hoping new topics will fix this.
New '-priority' queue is like '-bulk', but for smaller-volume SPN-like
requests. Eg, interactive mode.
Diffstat (limited to 'notes/url_pattern_heuristic_backfill.txt')
0 files changed, 0 insertions, 0 deletions