Commit message (Collapse) | Author | Age | Files | Lines | |
---|---|---|---|---|---|
* | persist: work around GROBID timeouts with no status_code | Bryan Newbold | 2020-01-21 | 1 | -1/+1 |
| | |||||
* | persist: fix dupe field copying | Bryan Newbold | 2020-01-15 | 1 | -1/+8 |
| | | | | | | In testing hit: AttributeError: 'str' object has no attribute 'get' | ||||
* | persist worker: implement updated ingest result semantics | Bryan Newbold | 2020-01-15 | 1 | -1/+1 |
| | |||||
* | small fixups to SandcrawlerPostgrestClient | Bryan Newbold | 2020-01-14 | 1 | -1/+10 |
| | |||||
* | db: move duplicate row filtering into DB insert helpers | Bryan Newbold | 2020-01-02 | 1 | -0/+25 |
| | |||||
* | fix DB import counting | Bryan Newbold | 2020-01-02 | 1 | -4/+5 |
| | |||||
* | fix small errors found by pylint | Bryan Newbold | 2020-01-02 | 1 | -1/+1 |
| | |||||
* | db: fancy insert/update separation using postgres xmax | Bryan Newbold | 2020-01-02 | 1 | -15/+30 |
| | |||||
* | improve DB helpers | Bryan Newbold | 2020-01-02 | 1 | -26/+81 |
| | | | | | - return insert/update row counts - implement ON CONFLICT ... DO UPDATE on some tables | ||||
* | start work on DB connector and minio client | Bryan Newbold | 2020-01-02 | 1 | -0/+141 |