1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
|
x search release, query string, limit, dumping search doc JSON
x search release, query string, limit, fetching API for each
x search release, query string, scroll API, fetching API for each
x handle stdout terminated
x editgroup creation
=> set agent
x editgroup accept
x editgroup submit
x editgroup list
x release create from json/TOML, to an editgroup
x release delete, to an editgroup
x release update from full json/TOML to API
x release edit (using $EDITOR, temp file)
x release update fields and submit to editgroup
=> more fields, like 2-5 for all entity types
x expand/hide flags for get, search
- search/update/etc containers (and files?)
- polish and test so actually usable for release edits from search
x consider moving to new repo, with copy of fatcat-openapi-client
x manpage
x .deb generation
=> write actual manpage (and, HTML output? ronn? pandoc?)
=> write actual README
- implement @-syntax for create/update
=> TODO: what was the proposal here?
=> some variant of @-syntax for stream of multiple updates/creations?
- get revisions for all entity types
#### Milestones
- ability (at all) to revise edits for a single entity in editgroup
=> clobber existing edits on update
=> edits: get entity in current edit state
- streaming updates from search, with either pipe (jq) or field mutations
=> syntax/commands
=> batching (syntax? subcommand?)
=> auto-accept mode
- download many PDFs from search query
=> parallelism could be GNU/parallel
=> don't clobber existing
#### Editgroup Workflow
- editgroup creation outputs just editgroup_id on stdout (unless output type selected), plus "success" to stderr
- parse editgroup specifier
=> "auto": fetch from recent; default?
=> "new": create
=> editgroup_blah or blah
- implement "delete from editgroup" for updates, edit
=> no updates with current setup
=> fetch editgroup helper
=> helper function that takes editgroup (model) and expanded specifier; deletes existing edit from editgroup if necessary
=> skip this codepath for "new" and batch creation
#### File Downloads
- download single file:
=> try archive.org files, then wayback, then original URLs
=> download to current directory as {sha1hex}.pdf.partial, then atomic move on success
- optional directory structure: {dir}/{hex}/{hex}/{sha1hex}.pdf
- parallelism of downloads
#### Backburner
- -o/--output and -i/--input for format/schema selection (including 'es-json')
- search release, filters, scroll API, fetching API for each
=> structopt parses: query, filter, anti-filter
- search release, filters, scroll API, fetching API for each, verifying revision and filters for each
## Design Decisions
- batch/multi behavior for mutations
=> need some option to do auto-accept batches
- updates and create, from-file vs. args
=> basically, could be any of specifier, input_file, mutations supplied on command-line
=> could use httpie @file.blah syntax to load entire file
=> "edit" as an option for reading single files from disk? meh
proposal:
create <type>
either reads a file from path/stdin, or has mutation args
optionally --new-editgroup
create-multi <type>
reads multiple JSON from file or stdin
optionally --auto-batch in chunks
optionally --new-editgroup
update <specifier>
takes a specifier
either reads a file from path/stdin, or has mutation args
update-multi <type>
reads multiple JSON from file or stdin
creates new editgroup?
edit <specifier>
delete <specifier>
delete-multi <type>
reads multiple entities from stdin
--skip-check controls whether to do a GET and validate mutations
=> eg, don't update if equal
- holding state about current editgroup
=> env var, with helpful output to show how to export
=> spawn sub-shell with FATCAT_EDITGROUP set
=> state in a config file somewhere (user homedir?)
=> "smart" select most recent fatcat-cli editgroup from editor's list
- release revision checking on updates
=> could re-fetch and check rev and/or mutations against current before making edit
- delete edit from editgroup
## Rust refactors
In rust code, all entity responses could have trait object implementations,
which would transform to either returning the entity (trait object) or error.
## API refactors
Could significantly reduce number of response types and endpoints by making
many methods generic (same endpoint URL, but entity type as a keyword):
- entity history
- delete
- get edit
Should allow destructive updates in editgroups with "clobber" flag. In
implementation, could either delete first or on conflict do upsert.
More consistent use of generic success/error?
## Feature Ideas
- changelog (table): under editgroup command?
- syntect coloring of output for stdout
- cross build for OS X? homebrew?
- shell (bash) completions from clap
- fcid/UUID helper
- history for all entity types
=> pretty table, json optional
- "edit editgroup" as a text file, `git rebase -i` style
|
