aboutsummaryrefslogtreecommitdiffstats
path: root/fatcat-cli
diff options
context:
space:
mode:
authorBryan Newbold <bnewbold@archive.org>2021-02-09 20:13:05 -0800
committerBryan Newbold <bnewbold@archive.org>2021-02-09 20:13:05 -0800
commit260ae4ec7ee4a836b35f036ead102be8347103e8 (patch)
treeaa044a914be9cb01a5fff287b558f5ff95cb59cf /fatcat-cli
parent0f89be721b820fe295cacfd5e3c875d3037b9874 (diff)
downloadfatcat-cli-260ae4ec7ee4a836b35f036ead102be8347103e8.tar.gz
fatcat-cli-260ae4ec7ee4a836b35f036ead102be8347103e8.zip
move around notes, docs, other text files
Diffstat (limited to 'fatcat-cli')
-rw-r--r--fatcat-cli/README.md2
-rw-r--r--fatcat-cli/binary_size.md88
-rw-r--r--fatcat-cli/fatcat-cli.164
-rw-r--r--fatcat-cli/fatcat-cli.1.scdoc48
-rw-r--r--fatcat-cli/plan.txt146
5 files changed, 0 insertions, 348 deletions
diff --git a/fatcat-cli/README.md b/fatcat-cli/README.md
deleted file mode 100644
index 1652d61..0000000
--- a/fatcat-cli/README.md
+++ /dev/null
@@ -1,2 +0,0 @@
-
-`fatcat-cli` README.
diff --git a/fatcat-cli/binary_size.md b/fatcat-cli/binary_size.md
deleted file mode 100644
index a79cf9b..0000000
--- a/fatcat-cli/binary_size.md
+++ /dev/null
@@ -1,88 +0,0 @@
-
-## Binary Size
-
-As of 2020-05-24, in early development, the relative binary sizes are:
-
- 121 MB default debug build
- 12 MB default release build
- 8.2 MB release build w/ LTO
- 6.6 MB release build w/ LTO, striped
-
-After some small changes:
-
- 5.9 MB release build w/ LTO, size optimization, other flags
- 4.1 MB release build w/ LTO, size optimization, other flags, striped
-
-Replacing reqwest with minreq:
-
- 6.3 MB release build w/ LTO, size optimization, other flags
- 4.1 MB release build w/ LTO, size optimization, other flags, striped
-
- (so, not worth it, at least while using fatcat_openapi with hyper+tokio)
-
-Note that release builds with LTO take *quite* a long time (many minutes). We
-probably don't want that to be the defualt for `fatcatd` builds.
-
- cargo bloat --release --crates
-
- File .text Size Crate
- 12.2% 21.4% 1021.5KiB fatcat_cli
- 7.1% 12.5% 596.7KiB fatcat_openapi
- 6.3% 11.1% 529.6KiB reqwest
- 6.2% 10.9% 518.5KiB std
- 3.5% 6.1% 290.3KiB clap
- 2.5% 4.3% 205.9KiB regex
- 2.4% 4.2% 198.7KiB regex_syntax
- 2.1% 3.6% 172.8KiB h2
- 1.9% 3.4% 162.7KiB hyper
- 1.8% 3.1% 149.9KiB futures
- 1.4% 2.4% 116.9KiB serde_json
- 1.3% 2.3% 111.2KiB macaroon
- 1.0% 1.8% 85.3KiB unicode_normalization
- 0.7% 1.3% 62.4KiB http
- 0.6% 1.0% 50.1KiB serde
- 0.6% 1.0% 47.5KiB url
- 0.5% 0.9% 41.9KiB [Unknown]
- 0.4% 0.8% 36.5KiB tokio_reactor
- 0.4% 0.7% 31.8KiB env_logger
- 0.3% 0.6% 26.6KiB chrono
- 3.4% 5.9% 283.3KiB And 57 more crates. Use -n N to show more.
- 57.2% 100.0% 4.7MiB .text section size, the file size is 8.2MiB
-
-
- bnewbold@orithena$ cargo bloat --release
- Finished release [optimized] target(s) in 0.27s
- Analyzing target/release/fatcat-cli
-
- File .text Size Crate Name
- 0.4% 1.0% 53.2KiB regex <regex::exec::ExecNoSync as regex::re_trait::RegularExpression>::capture...
- 0.4% 0.8% 44.1KiB regex_syntax regex_syntax::ast::parse::ParserI<P>::parse_with_comments
- 0.3% 0.7% 36.8KiB unicode_normalization unicode_normalization::tables::compatibility_fully_decomposed
- 0.3% 0.6% 30.3KiB unicode_normalization unicode_normalization::tables::canonical_fully_decomposed
- 0.2% 0.5% 25.2KiB data_encoding data_encoding::Encoding::decode_mut
- 0.2% 0.5% 24.0KiB fatcat_openapi? <fatcat_openapi::models::_IMPL_DESERIALIZE_FOR_ReleaseEntity::<impl serd...
- 0.2% 0.5% 23.5KiB clap clap::app::parser::Parser::get_matches_with
- 0.2% 0.4% 21.7KiB clap clap::app::validator::Validator::validate
- 0.2% 0.4% 20.6KiB http http::header::name::parse_hdr
- 0.2% 0.4% 19.5KiB fatcat_cli fatcat_cli::Specifier::get_from_api
- 0.1% 0.3% 16.4KiB fatcat_cli <&mut serde_json::de::Deserializer<R> as serde::de::Deserializer>::deser...
- 0.1% 0.3% 16.4KiB fatcat_cli <&mut serde_json::de::Deserializer<R> as serde::de::Deserializer>::deser...
- 0.1% 0.3% 16.2KiB fatcat_cli <&mut serde_json::de::Deserializer<R> as serde::de::Deserializer>::deser...
- 0.1% 0.3% 16.1KiB fatcat_cli fatcat_cli::run
- 0.1% 0.3% 15.2KiB fatcat_cli <&mut serde_json::de::Deserializer<R> as serde::de::Deserializer>::deser...
- 0.1% 0.3% 14.3KiB serde_json? <&mut serde_json::de::Deserializer<R> as serde::de::Deserializer>::deser...
- 0.1% 0.3% 14.2KiB fatcat_cli <&mut serde_json::de::Deserializer<R> as serde::de::Deserializer>::deser...
- 0.1% 0.3% 14.0KiB regex regex::exec::ExecBuilder::build
- 0.1% 0.3% 13.8KiB unicode_normalization unicode_normalization::tables::composition_table
- 0.1% 0.3% 13.6KiB fatcat_cli <&mut serde_json::de::Deserializer<R> as serde::de::Deserializer>::deser...
- 38.6% 89.5% 4.5MiB And 13832 smaller methods. Use -n N to show more.
- 43.1% 100.0% 5.1MiB .text section size, the file size is 11.8MiB
-
-Low hanging fruit includes:
-
-- reviewing features for reqwest, clap, regex, fatcat_openapi
-- replace reqwest with something smaller
-- use `ansi-term` (already part of clap)
-- consider removing fancy clap features? meh
-- look at graph; probably duplicate versions of things
-
diff --git a/fatcat-cli/fatcat-cli.1 b/fatcat-cli/fatcat-cli.1
deleted file mode 100644
index 0831773..0000000
--- a/fatcat-cli/fatcat-cli.1
+++ /dev/null
@@ -1,64 +0,0 @@
-.\" Generated by scdoc 1.9.0
-.ie \n(.g .ds Aq \(aq
-.el .ds Aq '
-.nh
-.ad l
-.\" Begin generated content:
-.TH "fatcat-cli" "1" "2020-06-14" "fatcat.wiki CLI Manual Page"
-.P
-.SH NAME
-.P
-fatcat-cli - something something
-.P
-.SH SYNOPSIS
-.P
-fatcat-cli [FLAGS] [OPTIONS] <SUBCOMMAND>
-.P
-Subcommands:
-.P
-.RS 4
-create
-delete
-edit
-editgroup
-get
-help
-search
-status
-update
-.P
-.RE
-.SH DESCRIPTION
-.P
-.SH OPTIONS
-.P
-\fB-h, --help\fR
-.RS 4
-Prints help information
-.P
-.RE
-\fB-V, --version\fR
-.RS 4
-Prints version information
-.P
-.RE
-\fB-v, --verbose\fR
-.RS 4
-Pass many times for more log output
-By default, it'll only report errors. Passing `-v` one time also prints warnings, `-vv` enables info logging, `-vvv` debug, and `-vvvv` trace.
-.P
-.RE
-\fB--api-host <api-host>\fR [env: FATCAT_API_HOST] [default: https://api.fatcat.wiki]
-.P
-\fB--api-token <api-token>\fR [env: FATCAT_API_AUTH_TOKEN]
-.P
-\fB--search-host <search-host>\fR [env: FATCAT_SEARCH_HOST] [default: https://search.fatcat.wiki]
-.P
-.SH EXAMPLES
-.P
-Some examples of usage will go here, like:
-.P
-.RS 4
-fatcat-cli get thing
-.P
-.RE
diff --git a/fatcat-cli/fatcat-cli.1.scdoc b/fatcat-cli/fatcat-cli.1.scdoc
deleted file mode 100644
index 6a176aa..0000000
--- a/fatcat-cli/fatcat-cli.1.scdoc
+++ /dev/null
@@ -1,48 +0,0 @@
-fatcat-cli(1) "fatcat.wiki CLI Manual Page"
-
-# NAME
-
-fatcat-cli - something something
-
-# SYNOPSIS
-
-fatcat-cli [FLAGS] [OPTIONS] <SUBCOMMAND>
-
-Subcommands:
-
- create
- delete
- edit
- editgroup
- get
- help
- search
- status
- update
-
-# DESCRIPTION
-
-# OPTIONS
-
-*-h, --help*
- Prints help information
-
-*-V, --version*
- Prints version information
-
-*-v, --verbose*
- Pass many times for more log output
- By default, it'll only report errors. Passing `-v` one time also prints warnings, `-vv` enables info logging, `-vvv` debug, and `-vvvv` trace.
-
-*--api-host <api-host>* [env: FATCAT_API_HOST] [default: https://api.fatcat.wiki]
-
-*--api-token <api-token>* [env: FATCAT_API_AUTH_TOKEN]
-
-*--search-host <search-host>* [env: FATCAT_SEARCH_HOST] [default: https://search.fatcat.wiki]
-
-# EXAMPLES
-
-Some examples of usage will go here, like:
-
- fatcat-cli get thing
-
diff --git a/fatcat-cli/plan.txt b/fatcat-cli/plan.txt
deleted file mode 100644
index 651acac..0000000
--- a/fatcat-cli/plan.txt
+++ /dev/null
@@ -1,146 +0,0 @@
-
-x search release, query string, limit, dumping search doc JSON
-x search release, query string, limit, fetching API for each
-x search release, query string, scroll API, fetching API for each
-
-x handle stdout terminated
-
-x editgroup creation
- => set agent
-x editgroup accept
-x editgroup submit
-x editgroup list
-
-x release create from json/TOML, to an editgroup
-x release delete, to an editgroup
-x release update from full json/TOML to API
-x release edit (using $EDITOR, temp file)
-
-x release update fields and submit to editgroup
- => more fields, like 2-5 for all entity types
-x expand/hide flags for get, search
-
-- search/update/etc containers (and files?)
-
-- polish and test so actually usable for release edits from search
- x consider moving to new repo, with copy of fatcat-openapi-client
- x manpage
- x .deb generation
- => write actual manpage (and, HTML output? ronn? pandoc?)
- => write actual README
-
-- implement @-syntax for create/update
- => TODO: what was the proposal here?
- => some variant of @-syntax for stream of multiple updates/creations?
-
-- get revisions for all entity types
-
-
-#### Milestones
-
-- ability (at all) to revise edits for a single entity in editgroup
- => clobber existing edits on update
- => edits: get entity in current edit state
-- streaming updates from search, with either pipe (jq) or field mutations
- => syntax/commands
- => batching (syntax? subcommand?)
- => auto-accept mode
-- download many PDFs from search query
- => parallelism could be GNU/parallel
- => don't clobber existing
-
-#### Editgroup Workflow
-
-- editgroup creation outputs just editgroup_id on stdout (unless output type selected), plus "success" to stderr
-- parse editgroup specifier
- => "auto": fetch from recent; default?
- => "new": create
- => editgroup_blah or blah
-- implement "delete from editgroup" for updates, edit
- => no updates with current setup
- => fetch editgroup helper
- => helper function that takes editgroup (model) and expanded specifier; deletes existing edit from editgroup if necessary
- => skip this codepath for "new" and batch creation
-
-#### File Downloads
-
-- download single file:
- => try archive.org files, then wayback, then original URLs
- => download to current directory as {sha1hex}.pdf.partial, then atomic move on success
-- optional directory structure: {dir}/{hex}/{hex}/{sha1hex}.pdf
-- parallelism of downloads
-
-#### Backburner
-
-- -o/--output and -i/--input for format/schema selection (including 'es-json')
-- search release, filters, scroll API, fetching API for each
- => structopt parses: query, filter, anti-filter
-- search release, filters, scroll API, fetching API for each, verifying revision and filters for each
-
-## Design Decisions
-
-- batch/multi behavior for mutations
- => need some option to do auto-accept batches
-- updates and create, from-file vs. args
- => basically, could be any of specifier, input_file, mutations supplied on command-line
- => could use httpie @file.blah syntax to load entire file
- => "edit" as an option for reading single files from disk? meh
- proposal:
- create <type>
- either reads a file from path/stdin, or has mutation args
- optionally --new-editgroup
- create-multi <type>
- reads multiple JSON from file or stdin
- optionally --auto-batch in chunks
- optionally --new-editgroup
- update <specifier>
- takes a specifier
- either reads a file from path/stdin, or has mutation args
- update-multi <type>
- reads multiple JSON from file or stdin
- creates new editgroup?
- edit <specifier>
- delete <specifier>
- delete-multi <type>
- reads multiple entities from stdin
-
- --skip-check controls whether to do a GET and validate mutations
- => eg, don't update if equal
-- holding state about current editgroup
- => env var, with helpful output to show how to export
- => spawn sub-shell with FATCAT_EDITGROUP set
- => state in a config file somewhere (user homedir?)
- => "smart" select most recent fatcat-cli editgroup from editor's list
-- release revision checking on updates
- => could re-fetch and check rev and/or mutations against current before making edit
-- delete edit from editgroup
-
-## Rust refactors
-
-In rust code, all entity responses could have trait object implementations,
-which would transform to either returning the entity (trait object) or error.
-
-## API refactors
-
-Could significantly reduce number of response types and endpoints by making
-many methods generic (same endpoint URL, but entity type as a keyword):
-
-- entity history
-- delete
-- get edit
-
-Should allow destructive updates in editgroups with "clobber" flag. In
-implementation, could either delete first or on conflict do upsert.
-
-More consistent use of generic success/error?
-
-## Feature Ideas
-
-- changelog (table): under editgroup command?
-- syntect coloring of output for stdout
-- cross build for OS X? homebrew?
-- shell (bash) completions from clap
-- fcid/UUID helper
-- history for all entity types
- => pretty table, json optional
-- "edit editgroup" as a text file, `git rebase -i` style