diff options
author | Bryan Newbold <bnewbold@archive.org> | 2021-02-09 20:13:05 -0800 |
---|---|---|
committer | Bryan Newbold <bnewbold@archive.org> | 2021-02-09 20:13:05 -0800 |
commit | 260ae4ec7ee4a836b35f036ead102be8347103e8 (patch) | |
tree | aa044a914be9cb01a5fff287b558f5ff95cb59cf /fatcat-cli | |
parent | 0f89be721b820fe295cacfd5e3c875d3037b9874 (diff) | |
download | fatcat-cli-260ae4ec7ee4a836b35f036ead102be8347103e8.tar.gz fatcat-cli-260ae4ec7ee4a836b35f036ead102be8347103e8.zip |
move around notes, docs, other text files
Diffstat (limited to 'fatcat-cli')
-rw-r--r-- | fatcat-cli/README.md | 2 | ||||
-rw-r--r-- | fatcat-cli/binary_size.md | 88 | ||||
-rw-r--r-- | fatcat-cli/fatcat-cli.1 | 64 | ||||
-rw-r--r-- | fatcat-cli/fatcat-cli.1.scdoc | 48 | ||||
-rw-r--r-- | fatcat-cli/plan.txt | 146 |
5 files changed, 0 insertions, 348 deletions
diff --git a/fatcat-cli/README.md b/fatcat-cli/README.md deleted file mode 100644 index 1652d61..0000000 --- a/fatcat-cli/README.md +++ /dev/null @@ -1,2 +0,0 @@ - -`fatcat-cli` README. diff --git a/fatcat-cli/binary_size.md b/fatcat-cli/binary_size.md deleted file mode 100644 index a79cf9b..0000000 --- a/fatcat-cli/binary_size.md +++ /dev/null @@ -1,88 +0,0 @@ - -## Binary Size - -As of 2020-05-24, in early development, the relative binary sizes are: - - 121 MB default debug build - 12 MB default release build - 8.2 MB release build w/ LTO - 6.6 MB release build w/ LTO, striped - -After some small changes: - - 5.9 MB release build w/ LTO, size optimization, other flags - 4.1 MB release build w/ LTO, size optimization, other flags, striped - -Replacing reqwest with minreq: - - 6.3 MB release build w/ LTO, size optimization, other flags - 4.1 MB release build w/ LTO, size optimization, other flags, striped - - (so, not worth it, at least while using fatcat_openapi with hyper+tokio) - -Note that release builds with LTO take *quite* a long time (many minutes). We -probably don't want that to be the defualt for `fatcatd` builds. - - cargo bloat --release --crates - - File .text Size Crate - 12.2% 21.4% 1021.5KiB fatcat_cli - 7.1% 12.5% 596.7KiB fatcat_openapi - 6.3% 11.1% 529.6KiB reqwest - 6.2% 10.9% 518.5KiB std - 3.5% 6.1% 290.3KiB clap - 2.5% 4.3% 205.9KiB regex - 2.4% 4.2% 198.7KiB regex_syntax - 2.1% 3.6% 172.8KiB h2 - 1.9% 3.4% 162.7KiB hyper - 1.8% 3.1% 149.9KiB futures - 1.4% 2.4% 116.9KiB serde_json - 1.3% 2.3% 111.2KiB macaroon - 1.0% 1.8% 85.3KiB unicode_normalization - 0.7% 1.3% 62.4KiB http - 0.6% 1.0% 50.1KiB serde - 0.6% 1.0% 47.5KiB url - 0.5% 0.9% 41.9KiB [Unknown] - 0.4% 0.8% 36.5KiB tokio_reactor - 0.4% 0.7% 31.8KiB env_logger - 0.3% 0.6% 26.6KiB chrono - 3.4% 5.9% 283.3KiB And 57 more crates. Use -n N to show more. - 57.2% 100.0% 4.7MiB .text section size, the file size is 8.2MiB - - - bnewbold@orithena$ cargo bloat --release - Finished release [optimized] target(s) in 0.27s - Analyzing target/release/fatcat-cli - - File .text Size Crate Name - 0.4% 1.0% 53.2KiB regex <regex::exec::ExecNoSync as regex::re_trait::RegularExpression>::capture... - 0.4% 0.8% 44.1KiB regex_syntax regex_syntax::ast::parse::ParserI<P>::parse_with_comments - 0.3% 0.7% 36.8KiB unicode_normalization unicode_normalization::tables::compatibility_fully_decomposed - 0.3% 0.6% 30.3KiB unicode_normalization unicode_normalization::tables::canonical_fully_decomposed - 0.2% 0.5% 25.2KiB data_encoding data_encoding::Encoding::decode_mut - 0.2% 0.5% 24.0KiB fatcat_openapi? <fatcat_openapi::models::_IMPL_DESERIALIZE_FOR_ReleaseEntity::<impl serd... - 0.2% 0.5% 23.5KiB clap clap::app::parser::Parser::get_matches_with - 0.2% 0.4% 21.7KiB clap clap::app::validator::Validator::validate - 0.2% 0.4% 20.6KiB http http::header::name::parse_hdr - 0.2% 0.4% 19.5KiB fatcat_cli fatcat_cli::Specifier::get_from_api - 0.1% 0.3% 16.4KiB fatcat_cli <&mut serde_json::de::Deserializer<R> as serde::de::Deserializer>::deser... - 0.1% 0.3% 16.4KiB fatcat_cli <&mut serde_json::de::Deserializer<R> as serde::de::Deserializer>::deser... - 0.1% 0.3% 16.2KiB fatcat_cli <&mut serde_json::de::Deserializer<R> as serde::de::Deserializer>::deser... - 0.1% 0.3% 16.1KiB fatcat_cli fatcat_cli::run - 0.1% 0.3% 15.2KiB fatcat_cli <&mut serde_json::de::Deserializer<R> as serde::de::Deserializer>::deser... - 0.1% 0.3% 14.3KiB serde_json? <&mut serde_json::de::Deserializer<R> as serde::de::Deserializer>::deser... - 0.1% 0.3% 14.2KiB fatcat_cli <&mut serde_json::de::Deserializer<R> as serde::de::Deserializer>::deser... - 0.1% 0.3% 14.0KiB regex regex::exec::ExecBuilder::build - 0.1% 0.3% 13.8KiB unicode_normalization unicode_normalization::tables::composition_table - 0.1% 0.3% 13.6KiB fatcat_cli <&mut serde_json::de::Deserializer<R> as serde::de::Deserializer>::deser... - 38.6% 89.5% 4.5MiB And 13832 smaller methods. Use -n N to show more. - 43.1% 100.0% 5.1MiB .text section size, the file size is 11.8MiB - -Low hanging fruit includes: - -- reviewing features for reqwest, clap, regex, fatcat_openapi -- replace reqwest with something smaller -- use `ansi-term` (already part of clap) -- consider removing fancy clap features? meh -- look at graph; probably duplicate versions of things - diff --git a/fatcat-cli/fatcat-cli.1 b/fatcat-cli/fatcat-cli.1 deleted file mode 100644 index 0831773..0000000 --- a/fatcat-cli/fatcat-cli.1 +++ /dev/null @@ -1,64 +0,0 @@ -.\" Generated by scdoc 1.9.0 -.ie \n(.g .ds Aq \(aq -.el .ds Aq ' -.nh -.ad l -.\" Begin generated content: -.TH "fatcat-cli" "1" "2020-06-14" "fatcat.wiki CLI Manual Page" -.P -.SH NAME -.P -fatcat-cli - something something -.P -.SH SYNOPSIS -.P -fatcat-cli [FLAGS] [OPTIONS] <SUBCOMMAND> -.P -Subcommands: -.P -.RS 4 -create -delete -edit -editgroup -get -help -search -status -update -.P -.RE -.SH DESCRIPTION -.P -.SH OPTIONS -.P -\fB-h, --help\fR -.RS 4 -Prints help information -.P -.RE -\fB-V, --version\fR -.RS 4 -Prints version information -.P -.RE -\fB-v, --verbose\fR -.RS 4 -Pass many times for more log output -By default, it'll only report errors. Passing `-v` one time also prints warnings, `-vv` enables info logging, `-vvv` debug, and `-vvvv` trace. -.P -.RE -\fB--api-host <api-host>\fR [env: FATCAT_API_HOST] [default: https://api.fatcat.wiki] -.P -\fB--api-token <api-token>\fR [env: FATCAT_API_AUTH_TOKEN] -.P -\fB--search-host <search-host>\fR [env: FATCAT_SEARCH_HOST] [default: https://search.fatcat.wiki] -.P -.SH EXAMPLES -.P -Some examples of usage will go here, like: -.P -.RS 4 -fatcat-cli get thing -.P -.RE diff --git a/fatcat-cli/fatcat-cli.1.scdoc b/fatcat-cli/fatcat-cli.1.scdoc deleted file mode 100644 index 6a176aa..0000000 --- a/fatcat-cli/fatcat-cli.1.scdoc +++ /dev/null @@ -1,48 +0,0 @@ -fatcat-cli(1) "fatcat.wiki CLI Manual Page" - -# NAME - -fatcat-cli - something something - -# SYNOPSIS - -fatcat-cli [FLAGS] [OPTIONS] <SUBCOMMAND> - -Subcommands: - - create - delete - edit - editgroup - get - help - search - status - update - -# DESCRIPTION - -# OPTIONS - -*-h, --help* - Prints help information - -*-V, --version* - Prints version information - -*-v, --verbose* - Pass many times for more log output - By default, it'll only report errors. Passing `-v` one time also prints warnings, `-vv` enables info logging, `-vvv` debug, and `-vvvv` trace. - -*--api-host <api-host>* [env: FATCAT_API_HOST] [default: https://api.fatcat.wiki] - -*--api-token <api-token>* [env: FATCAT_API_AUTH_TOKEN] - -*--search-host <search-host>* [env: FATCAT_SEARCH_HOST] [default: https://search.fatcat.wiki] - -# EXAMPLES - -Some examples of usage will go here, like: - - fatcat-cli get thing - diff --git a/fatcat-cli/plan.txt b/fatcat-cli/plan.txt deleted file mode 100644 index 651acac..0000000 --- a/fatcat-cli/plan.txt +++ /dev/null @@ -1,146 +0,0 @@ - -x search release, query string, limit, dumping search doc JSON -x search release, query string, limit, fetching API for each -x search release, query string, scroll API, fetching API for each - -x handle stdout terminated - -x editgroup creation - => set agent -x editgroup accept -x editgroup submit -x editgroup list - -x release create from json/TOML, to an editgroup -x release delete, to an editgroup -x release update from full json/TOML to API -x release edit (using $EDITOR, temp file) - -x release update fields and submit to editgroup - => more fields, like 2-5 for all entity types -x expand/hide flags for get, search - -- search/update/etc containers (and files?) - -- polish and test so actually usable for release edits from search - x consider moving to new repo, with copy of fatcat-openapi-client - x manpage - x .deb generation - => write actual manpage (and, HTML output? ronn? pandoc?) - => write actual README - -- implement @-syntax for create/update - => TODO: what was the proposal here? - => some variant of @-syntax for stream of multiple updates/creations? - -- get revisions for all entity types - - -#### Milestones - -- ability (at all) to revise edits for a single entity in editgroup - => clobber existing edits on update - => edits: get entity in current edit state -- streaming updates from search, with either pipe (jq) or field mutations - => syntax/commands - => batching (syntax? subcommand?) - => auto-accept mode -- download many PDFs from search query - => parallelism could be GNU/parallel - => don't clobber existing - -#### Editgroup Workflow - -- editgroup creation outputs just editgroup_id on stdout (unless output type selected), plus "success" to stderr -- parse editgroup specifier - => "auto": fetch from recent; default? - => "new": create - => editgroup_blah or blah -- implement "delete from editgroup" for updates, edit - => no updates with current setup - => fetch editgroup helper - => helper function that takes editgroup (model) and expanded specifier; deletes existing edit from editgroup if necessary - => skip this codepath for "new" and batch creation - -#### File Downloads - -- download single file: - => try archive.org files, then wayback, then original URLs - => download to current directory as {sha1hex}.pdf.partial, then atomic move on success -- optional directory structure: {dir}/{hex}/{hex}/{sha1hex}.pdf -- parallelism of downloads - -#### Backburner - -- -o/--output and -i/--input for format/schema selection (including 'es-json') -- search release, filters, scroll API, fetching API for each - => structopt parses: query, filter, anti-filter -- search release, filters, scroll API, fetching API for each, verifying revision and filters for each - -## Design Decisions - -- batch/multi behavior for mutations - => need some option to do auto-accept batches -- updates and create, from-file vs. args - => basically, could be any of specifier, input_file, mutations supplied on command-line - => could use httpie @file.blah syntax to load entire file - => "edit" as an option for reading single files from disk? meh - proposal: - create <type> - either reads a file from path/stdin, or has mutation args - optionally --new-editgroup - create-multi <type> - reads multiple JSON from file or stdin - optionally --auto-batch in chunks - optionally --new-editgroup - update <specifier> - takes a specifier - either reads a file from path/stdin, or has mutation args - update-multi <type> - reads multiple JSON from file or stdin - creates new editgroup? - edit <specifier> - delete <specifier> - delete-multi <type> - reads multiple entities from stdin - - --skip-check controls whether to do a GET and validate mutations - => eg, don't update if equal -- holding state about current editgroup - => env var, with helpful output to show how to export - => spawn sub-shell with FATCAT_EDITGROUP set - => state in a config file somewhere (user homedir?) - => "smart" select most recent fatcat-cli editgroup from editor's list -- release revision checking on updates - => could re-fetch and check rev and/or mutations against current before making edit -- delete edit from editgroup - -## Rust refactors - -In rust code, all entity responses could have trait object implementations, -which would transform to either returning the entity (trait object) or error. - -## API refactors - -Could significantly reduce number of response types and endpoints by making -many methods generic (same endpoint URL, but entity type as a keyword): - -- entity history -- delete -- get edit - -Should allow destructive updates in editgroups with "clobber" flag. In -implementation, could either delete first or on conflict do upsert. - -More consistent use of generic success/error? - -## Feature Ideas - -- changelog (table): under editgroup command? -- syntect coloring of output for stdout -- cross build for OS X? homebrew? -- shell (bash) completions from clap -- fcid/UUID helper -- history for all entity types - => pretty table, json optional -- "edit editgroup" as a text file, `git rebase -i` style |