sandcrawler/sql, branch bnewbold-refactor-loggging

sandcrawler/sql, branch bnewbold-refactor-loggging [no description] https://git.bnewbold.net/sandcrawler/atom?h=bnewbold-refactor-loggging 2022-07-07T20:19:12+00:00 stats: may 2022 ingest-by-domain stats 2022-07-07T20:19:12+00:00 Bryan Newbold bnewbold@archive.org 2022-07-07T20:19:12+00:00 urn:sha1:bf1826f8e8d203f732cbdda008e0c5944cbdae60 some weekly crawl numbers (not very helpful) 2022-05-04T00:13:30+00:00 Bryan Newbold bnewbold@archive.org 2022-05-04T00:13:30+00:00 urn:sha1:6dd9bc8d3312107796344341e43044907677bf85 switch default kafka-broker host from wbgrp-svc263 to wbgrp-svc350 2022-05-04T00:12:48+00:00 Bryan Newbold bnewbold@archive.org 2022-05-04T00:12:48+00:00 urn:sha1:00ae74378413e87f230c88113ff8163a6f969d63 April 2022 sandcrawler DB stats 2022-04-27T18:55:41+00:00 Bryan Newbold bnewbold@archive.org 2022-04-27T18:55:41+00:00 urn:sha1:ef0421567dd67a248d0f92f32ad4e14ae0776920 sql: add source/created index on ingest_request table 2022-04-04T22:52:02+00:00 Bryan Newbold bnewbold@archive.org 2022-04-04T22:52:02+00:00 urn:sha1:dadb26935c4d255c5a662f1e758bcf53864f7f95 sql: fix reingest query missing type on LEFT JOIN; wrap in read-only transaction 2022-04-04T22:27:02+00:00 Bryan Newbold bnewbold@archive.org 2022-04-04T22:27:02+00:00 urn:sha1:c4caeaedc63c2bb2f9bb9f6be6826480aefc9caf sql: script to reingest recent spn2 lookup failure in bulk mode 2022-02-09T00:57:16+00:00 Bryan Newbold bnewbold@archive.org 2022-02-09T00:57:16+00:00 urn:sha1:b76630806d43427ddcdef8cd6efd110c731668a5 2021-12-02 database table size stats 2021-12-08T03:10:23+00:00 Bryan Newbold bnewbold@archive.org 2021-12-04T00:37:45+00:00 urn:sha1:57441fda8be33594898c1836fba22b12fb3e94e8 sandcrawler SQL dump and upload updates 2021-12-08T03:10:23+00:00 Bryan Newbold bnewbold@archive.org 2021-12-04T00:37:22+00:00 urn:sha1:9a32daa502e2c729cf896ae5e7cb27a3aa6bb68d update fatcat_file SQL table schema, and add backfill notes 2021-12-08T03:10:23+00:00 Bryan Newbold bnewbold@archive.org 2021-12-02T03:06:33+00:00 urn:sha1:3f01f73563f40869c82b7ad3e21c4183fdee8206