aboutsummaryrefslogtreecommitdiffstats
path: root/python/sandcrawler/ingest_fileset.py
Commit message (Collapse)AuthorAgeFilesLines
* small lint/typo/fmt fixesBryan Newbold2022-02-241-3/+3
|
* ingest: fix mistakenly commented except block (?)Bryan Newbold2022-02-181-4/+3
|
* ingest: handle more fileset failure modesBryan Newbold2022-02-181-2/+25
|
* null-body -> empty-blobBryan Newbold2022-01-131-2/+2
|
* fileset ingest: more requests timeouts, sessionsBryan Newbold2021-12-151-1/+7
|
* fileset ingest: actually use spn2 CLI flagBryan Newbold2021-12-111-2/+3
|
* make fmt (black 21.9b0)Bryan Newbold2021-10-271-165/+207
|
* fileset: refactor out tables of helpersBryan Newbold2021-10-271-5/+19
| | | | | | | Having these objects invoked in tables resulted in a whole bunch of objects (including children) getting initialized, which seems like the wrong thing to do. Defer this until the actual ingest fileset worker is initialized.
* small type annotation hackBryan Newbold2021-10-261-1/+1
|
* fileset: fix field renaming bug (caught by mypy)Bryan Newbold2021-10-261-2/+2
|
* fileset ingest: fix table name typo (via mypy)Bryan Newbold2021-10-261-1/+1
|
* update 'XXX' notes from fileset ingest developmentBryan Newbold2021-10-261-1/+1
|
* bugfix: setting html_biblio on ingest resultsBryan Newbold2021-10-261-1/+1
| | | | This was caught during lint cleanup
* ingest fileset: fix silly import typoBryan Newbold2021-10-261-1/+1
|
* more progress on type annotationsBryan Newbold2021-10-261-2/+3
|
* flake8 clean (with current settings)Bryan Newbold2021-10-261-6/+2
|
* start handling trivial lint cleanups: unused imports, 'is None', etcBryan Newbold2021-10-261-20/+9
|
* make fmtBryan Newbold2021-10-261-41/+60
|
* python: isort all importsBryan Newbold2021-10-261-16/+15
|
* more fileset iterationBryan Newbold2021-10-151-33/+56
|
* filesets: iteration of implementation and docsBryan Newbold2021-10-151-34/+89
|
* fileset ingest: improve error handlingBryan Newbold2021-10-151-7/+40
|
* improvements to platform helpersBryan Newbold2021-10-151-1/+1
|
* progress on web ingest strategyBryan Newbold2021-10-151-2/+7
|
* fileset ingest progress for dataverseBryan Newbold2021-10-151-6/+8
|
* progress on dataset ingestBryan Newbold2021-10-151-108/+118
|
* progress on fileset/dataset ingestBryan Newbold2021-10-151-0/+299