diff options
author | Bryan Newbold <bnewbold@robocracy.org> | 2018-05-23 23:40:10 -0700 |
---|---|---|
committer | Bryan Newbold <bnewbold@robocracy.org> | 2018-05-24 15:21:32 -0700 |
commit | 1b4892a29889ffd8013a720aa7ba986ae0809485 (patch) | |
tree | c8aefbedd6a97091dc99ace994ff5fe676da6e6d /notes | |
parent | 9479b3389907f3844a13d5e6cf881e27901b5aab (diff) | |
download | fatcat-1b4892a29889ffd8013a720aa7ba986ae0809485.tar.gz fatcat-1b4892a29889ffd8013a720aa7ba986ae0809485.zip |
update speed notes
Diffstat (limited to 'notes')
-rw-r--r-- | notes/speed.txt | 31 |
1 files changed, 31 insertions, 0 deletions
diff --git a/notes/speed.txt b/notes/speed.txt index 69be3253..86561a80 100644 --- a/notes/speed.txt +++ b/notes/speed.txt @@ -42,3 +42,34 @@ same except postgres, via: no profiling yet; need to look at database ops. probably don't even have any indices! + +## Rust Updates (2018-05-23) + +Re-running with tweaked python code, 5k sample file, postgres 9.6 running locally (not in docker): + + real 2m27.598s + user 0m24.892s + sys 0m2.836s + +Using postgres and fatcat rust: + + real 0m44.443s + user 0m25.288s + sys 0m0.880s + +api_client about half a core; fatcatd 3x processes, about 10% each; postgres +very small. + +a bit faster, basically maxing out CPU: + + time cat /data/crossref/crossref-works.2018-01-21.badsample_5k.json | parallel -j4 --pipe ./fatcat_client.py --host-url http://localhost:9411 ic - + + real 0m28.998s + user 1m5.304s + sys 0m3.420s + + 200 lines per second; within a factor of 3; can perhaps hit target with + non-python client? + +python processes (clients) seem to be CPU limit in this case; all 4 cores +effectively maxed out. |