summaryrefslogtreecommitdiffstats
path: root/python
diff options
context:
space:
mode:
authorBryan Newbold <bnewbold@robocracy.org>2018-06-30 23:33:03 -0700
committerBryan Newbold <bnewbold@robocracy.org>2018-06-30 23:33:03 -0700
commit5e2fecf4b81a878ec4321cdd85d6a594e94c1eb2 (patch)
treeba0b0affbff2f1c41e22a9ec0a548e8e80b06bb3 /python
parent224e4929af6fa84401944c757d9902a5f83da7a3 (diff)
downloadfatcat-5e2fecf4b81a878ec4321cdd85d6a594e94c1eb2.tar.gz
fatcat-5e2fecf4b81a878ec4321cdd85d6a594e94c1eb2.zip
update TODO/notes again
Diffstat (limited to 'python')
-rw-r--r--python/README_import.md13
1 files changed, 13 insertions, 0 deletions
diff --git a/python/README_import.md b/python/README_import.md
index f43d9424..ae9764e6 100644
--- a/python/README_import.md
+++ b/python/README_import.md
@@ -99,3 +99,16 @@ From compressed:
## Manifest
time ./client.py import-manifest /srv/datasets/idents_files_urls.sqlite
+
+ [...]
+ Finished a batch; row 284518671 of 9669646 (2942.39%). Total inserted: 6606900
+ Finished a batch; row 284518771 of 9669646 (2942.39%). Total inserted: 6606950
+ Finished a batch; row 284518845 of 9669646 (2942.39%). Total inserted: 6607000
+ Finished a batch; row 284518923 of 9669646 (2942.39%). Total inserted: 6607050
+ Done! Inserted 6607075
+
+ real 1590m36.626s
+ user 339m40.928s
+ sys 19m3.576s
+
+Really sped up once not contending with Crossref import, so don't run these two at the same time.