Merge branch 'master' of adze:knowledge

13 files changed, 553 insertions, 16 deletions

diff --git a/books/2019.page b/books/2019.page
new file mode 100644
index 0000000..762e1bf
--- /dev/null
+++ b/books/2019.page
@@ -0,0 +1,168 @@
+
+Cataloging The World: Paul Otlet and the Birth of the Information Age, by Alex Wright
+-----------------------------------------------------------------------------------------
+
+> The huge mass of published material grows by the day, by the hour, in
+> amounts that are disconcerting and sometimes maddening. Like water falling
+> from the sky, it can either cause flooding or beneficial irrigation
+
+I loved this book!
+
+Notes while reading:
+
+- "Biblion" as a unit of writing (and knowledge).
+- Embodied Cognition
+
+
+Singlularity Sky, by Charlie Stross
+--------------------------------------
+
+Had I really not read this? Maybe and forgot. Such strong optimism for info
+maximalism and info-structures. Characters and writing meh; mostly interesting
+for the taste of period (cyber)idiology.
+
+Overall, standard 90s singularity/space-opera genre fare.
+
+
+Dark Matter, by Blake Crouch (2016)
+--------------------------------------
+
+Simple book, pretty well executed. Read like a film script, or a TV episode,
+but with more twists. I liked the last quarter; much of the early exposition
+was very slow and predictable. Good balance of fine details while glossing over
+some hard physics which could have been an over-reach.
+
+
+Oranges, by John McPhee
+---------------------------
+
+Ate so many oranges after reading this. Cara Caras are great, but had some
+incredibly juicy flavorful oranges with Lucy at the kitchen table that now are
+driving me mad that I can't remember the type. Changed my standards a lot: many
+navels are great, many other easy-to-peel don't actually have much flavor.
+
+Orangeries! Florida!
+
+I like the small bit of 4th wall that McPhee breaks.
+
+
+The World of Edena, by Moebius
+----------------------------------
+
+Always such a feeling of boundless creative universe with Moebius; could just
+go on forever. Feels dated in a sometimes uncomfortable way (lots of naked
+ladies), but also fresh and humanist.
+
+
+The City and The City, by China Meville
+-----------------------------------------
+
+For whatever reason I was skeptical going in... too popular? Too heavy-handed a
+gimick? But liked it immediately, both the structure and the
+characters/exposition. Not super happy with the resolution of the mystery, but
+very happy with how the character arcs ended.
+
+
+Broken Earth Trilogy by N. K. Jemisin (2015-2017)
+-----------------------------------------------------------------------------
+
+Oof, I binge-read this trilogy (**The Fifth Season**, **The Obelisk Gate**, and
+**The Stone Sky** in one week, which wasn't particularly healthy, and wasn't
+mindful or thoughtful.
+
+The books were tightly written and well paced. I mostly liked the characters,
+but the "world building" and exposition felt like the real show here. The mix
+of magical realism and sci-fi worked surprisingly well to me, though I think I
+prefered the fuzzy-but-hard science of Anathem (by N. Stephenson) more.
+Surprised how fascinated in the "orogenes" power/curse I was.
+
+Overall well written and different. During and after I keep thinking of this as
+young-adult or genre entertainment reading; there's more to it than that, but
+also less than more traditional adult literature.
+
+
+Energy by Richard Rhodes (201?)
+--------------------------------
+
+After "Making of the Atomic Bomb", a bit of a narrative disapointment, though
+it is just a different sort of book. Felt like a series of snapshots, none deep
+enough to feel like I really understood the course and pressures that lead to
+success of different energy technologies.
+
+An over-arching theme was that ideas were had well before acceptance; it was
+often a combination of small technical polish *and* external economic or
+political changes that led to a new source being adopted.
+
+Narrative of coal, steam engine, and trains being intertwined was interesting:
+coal nominally being used as a heating source, but required engines for
+economical mining and transport; the engines themselves requiring cheap coal to
+be worth developing. And along the way land-use regulation being a blocker.
+
+Surprising to hear how much the negative health impacts of fossil fuels were
+known from the begining, and how bad the (local) environmental impacts were.
+The global impact gets so much more attention today. The period belief from the
+start that oil and coal reserves would run out. How poor Saudi Arabia was, and
+how narrowly the kingdom survived by oil exploration taking off at just the
+right moment.
+
+Part of what makes Niagra such a great power location is that the lake it
+drains is a huge buffer of stored water (thus energy), and the flow rate can be
+controlled at will (no flooding). More than a year of reserve water at full
+full (including the fact that water level would be decreasing).
+
+Didn't know that religious minorities on Nantucket partially moved back to
+Europe at some point to continue to pursue whaling.
+
+
+Roadside Picnic
+------------------
+
+Oh, I really loved this. Very Russian. Explains "Stalker" the same way "2001: A
+Space Odessy" makes sense if you read the script/narration.
+
+The informal/intimate stalkers against the official/institutional scientists
+were so spot-on. This pattern doesn't always hold in sci/tech world, but it is
+pretty common.
+
+
+Devil and the White City by Eric Larson
+-------------------------------------------
+
+Decent, easy flight reading. Focus on the serial killer thread is of course
+only on the principle actors, but in the case of the fair, the focus on a
+handful of leaders and planners was less compelling.
+
+The scale of the Fair as a singular and super-human event really comes through.
+Will this sort of economic activity and make-work become more popular during
+late capitalism? Or post-scarcity? I continue to be perplexed why the scale of
+architecture gets less ambitious as society becomes more technically powerful;
+was it really dependent on economic inequality and exploitation of labor? Don't
+we have that again today?
+
+The background of economic recession, homelessness, and desparation against the
+robber barons funding and directing the World's Faire seemed like the real
+story and didn't get much coverage in depth.
+
+Combined with "Cadillac Desert", paints a story of agricultural development of
+the American mid-west as an economic and policy tragedy of the same
+incompetence as Soviet/Mao-ist economic planning, though of course far less of
+a tragedy in the end as most were able to survive and freely relocated.
+
+
+The Overstory by Richard Powers
+--------------------------------
+
+Decent, not spectacular. Most of the individual story threads would not have
+stood well on their own. The tree protectors were the most compelling to me:
+the aimless artist with a family flipbook of great tree growth, and the
+near-death college dropout. The various endings are pretty dramatic.
+
+Had echos of "The Wizard and the Prophet".
+
+
+Gandhi
+----------
+
+Easy read; very basic introduction to the person and this period in history.
+Read because even this much I did not know!
+
diff --git a/books/2020.page b/books/2020.page
new file mode 100644
index 0000000..16a321f
--- /dev/null
+++ b/books/2020.page
@@ -0,0 +1,33 @@
+
+
+Tai Pan
+========
+
+
+The Nobel House
+=================
+
+
+Patience and Fortitude
+========================
+
+
+Looking For a Ship
+=====================
+
+
+Edition of One (Eugene Powers, University Microfilms)
+=========================================================
+
+WWII; intelligence work.
+
+ARL meeting sealed fate as sole provider of dissertation publishing in USA.
+
+Regant of U. Michigan.
+
+
+The Moon is a Harsh Mistress
+================================
+
+Decent.
+
diff --git a/books/to-read.page b/books/to-read.page
index c8b5f3c..72cc4ff 100644
--- a/books/to-read.page
+++ b/books/to-read.page
@@ -10,9 +10,9 @@ Novels
 * The Corrections by Jonathan Franzen
 * The Magic Mountain, thomas mann
 * Grapes of Wrath
-* The Illuminatus! Trilogy
 * The Sympathizer by Viet Thanh Nguyen
-* Red Plenty (historical fiction)
+* Midnght's Children by Rushdie
+* Big Sur by Jack Kerouac
 
 Old Classics: 
 
@@ -29,6 +29,8 @@ Science/Tech
 * War in the Age of Intelligent Machines by Manuel De Landa
 * Nonlinear Time Series Analysis, Holger Kantz Thomas Schreiber
 * Field Notes on Science & Nature
+* The Nature of Mathematical Modeling
+* Spacetime and Geometry by Sean Carroll
 
 Philosophy
 ===============
@@ -44,9 +46,12 @@ Philosophy
 History and Politics
 =======================
 
+* Origins of Totalitarianism by Arendt
+* Swaraj, Gandhi
 * Reflections on the Revolution in France by Edmund Burke
 * The Art of Intelligence by Henry Crumpton (CIA history)
 * The Conscience of a Conservative "by" Barry Goldwater
+* The City in History by Lewis Mumfordk
 
 Chinese History:
 
@@ -58,5 +63,6 @@ Other Non-Fiction
 =====================
 
 * Looking for a Ship, John McPhee (merchant marine)
-* Uncommon Carriers, John McPhee (shipping)
-* The Making of the Atomic Bomb by Richard Rhodes
+* The Educated Mind: How Cognitive Tools Shape Our Understanding
+* The Feminine Mystique by Betty Friedan
+* A Sand Country Almanac by Aldo Leopold
diff --git a/film/2018.page b/film/2018.page
index 1aae06f..af6694c 100644
--- a/film/2018.page
+++ b/film/2018.page
@@ -41,7 +41,7 @@ I would take 8 1/2 over this film in a heartbeat.
 Infernal Affairs II
 ---------------------
 
-Saw it on the plane and don't remember much, which was a losss.
+Saw it on the plane and don't remember much, which was a loss.
 
 Die Hard
 --------
diff --git a/film/2019.page b/film/2019.page
new file mode 100644
index 0000000..cec3ab9
--- /dev/null
+++ b/film/2019.page
@@ -0,0 +1,91 @@
+
+
+"Recent Godard Film" (can't remember title)
+---------------------------------------------
+
+This was a "hard" serious film, and to be honest I didn't follow a single
+minute of it. I think rmo, who I saw it with, saw and connected much more, but
+even his explanations didn't make any sense to me. Still not sure if
+disapointed in myself (for not knowing more context?) or the film (for being
+inscrutiable, and/or indulgent and vapid). Reminded me of the Orson Wells film
+from last year.
+
+
+John Wick 3
+--------------
+
+Saw with mouse; I think most of my enjoyment came as spill over. The library
+scene was fun, the desert shootout with doggies was gripping. The degree of of
+style but total ridiculousness is new to me. I liked the type-writer
+score-keeping room entirely run by punk (women?) in starched white shirts.
+
+
+Southland Tales
+-----------------
+
+Re-watched this at home in SF with Lucy and Will. They didn't love it, but
+could stomach it, I think. Had recently visited LA and it was sort of fun to
+remember real locations. Just after screening I felt disapointed and over it
+(heavy-handed, long, slow, plot is a mess, gag-oriented, indulgent, whatever),
+but now months later I still feel like I love the film for it's weirdness, the
+musical scene, and Dwayne Johnson.
+
+
+"Flamboyant Portugese Film" (can't remember title)
+---------------------------------------------------
+
+Mixed/weird feelings about this one. It wasn't very good overall, lots of slow
+or "ugh" jokes and moments, but there were enough surreal visual gags to be
+compeling.
+
+I liked the undercover character, and of course the pink puppy football
+sequences. Having it be another culture (Portugal) and language made it easier
+to laugh, though very close to the bone (Trump era).
+
+
+Crappy MCU Films
+-----------------
+
+Guess I watched a bunch of these this spring? Final Avengers film, Captain
+America? Can't even remember now. I love a summer blockbuster; I fondly
+remember escaping NYC heat and humidity for giant robot battles, and the
+spectacle of Cinerama action films. But feel like the pattern of shutting off
+higher brain function and letting the high-production-value slurry drain down
+has become addictive and un-fun.
+
+
+The Farewell (2019)
+---------------------
+
+Saw with Lucy at the Egyptian in Seattle. A good film, would recommend in
+general, but it particularly touched both Lucy and me for being so close to her
+personal family story and travel experiences in China (both her own trips and
+us together to see her family). The theater was full of couples in tears.
+
+I liked this so much more than "Crazy Rich Asians", though it probably won't
+reach as wide an audience.
+
+
+Only God Forgives
+--------------------
+
+Meh.
+
+
+Towering Inferno
+--------------------
+
+SF Hyatt Regency, OJ Simpson, etc!
+
+
+I am Love
+-------------
+
+Solid family drama. Always love Tilda Swinton in anything.
+
+
+American Psycho
+-------------------
+
+Alright I guess, only watched for the pop culture.
+
diff --git a/film/2020.page b/film/2020.page
new file mode 100644
index 0000000..611a5ef
--- /dev/null
+++ b/film/2020.page
@@ -0,0 +1,59 @@
+
+The Net (1995)
+----------------
+
+Subway
+-----------
+
+Fun, aesthetic, but didn't feel like there was much there there. Sort of
+reminded me of Buckaroo Banzai.
+
+
+Pain and Glory (Almadovar)
+-------------------------------------
+
+This was great! Hard to put a finger on why exactly I liked it so much though.
+Obviously much less tranditionally masculine than 8 1/2.
+
+
+Bullitt
+--------------
+
+This has been on my list since J screened "Green Fog" for me at the archive,
+for the San Francisco setting.  McQueen is swaggery, though I expected even
+more aggression from the reputation the film has. The chase scene is wild and
+great. The cafe and SF culture establishing shots were unexpected.
+
+To be honest had to check some of the plot points online after... screen we
+watched on didn't have good audio.
+
+I watched "Towering Inferno" before this, partially for the same "Green Fog"
+motivation, and partially because it features the Embarcadero Hyatt atrium.
+Also features McQueen and a couple other scenes were similar (everybody in
+Pacific Heights is rich and horrible), but Bullitt is much better.
+
+Those blue McQueen eyes!
+
+
+Pain and Glory
+-----------------
+
+Good.
+
+
+How to Get Ahead in Advertising
+---------------------------------
+
+I didn't like this as much as I expected I would? Manic, ranty, nice Cronberg-y
+bits.
+
+The Lighthouse
+------------------
+
+Weird, intense. Good? Acting was great. Dialog was hard to understand at times.
+
+
+Bombshell
+-------------
+
+This was good. A juicy, buzzy story.
diff --git a/film/to-watch.page b/film/to-watch.page
index 4b56159..37223bd 100644
--- a/film/to-watch.page
+++ b/film/to-watch.page
@@ -57,7 +57,7 @@ Films To Watch
  * American Psycho
  * Network
  * The Tree of Life
- ! Glengarry Glen Ross
+ * Glengarry Glen Ross
  * The Man Who Wasn't There (2001, Coen Brothers)
  * House of Games
  * We Children from Bahnhof Zoo (Christiane F)
@@ -120,7 +120,15 @@ Films To Watch
  * Woman on Top (Penelope Cruz romcom, 2000)
  * Okja (2017)
  * All About Eve (1950)
+ * The Founder (McDonalds; 2017)
+ * Wax or the Discovery of Television Among the Bees (1991; net art)
+ * Song of the Exile (1990, HK, Maggie Cheung)
+ * Center Stage (1991, HK, Maggie Cheung)
+ * Clean (Assayas, Maggie Cheung)
  * The Big Chill (Jeff Goldblum)
+ * Hunger (2008, McQueen)
+ * Adults in the Room (2019; Yanis Varofakis)
+ * Virtuosity (1995)
  * Your Name (2016, animated; maybe?)
  * Boyz n the Hood (1991)
  * Mr Smith Goes to Washington (1939)
@@ -150,6 +158,7 @@ Documentaries:
  * The Kingdom of Dreams and Madness (2014; Gibli+Miyazaki)
  * Ex Libris (NYC Library)
  * Sinai Field Mission
+ * The Art of the Steal (2009; art collection)
 
 Other Lists:
 
diff --git a/networking/bandwidth.page b/networking/bandwidth.page
index f516087..77fd5ed 100644
--- a/networking/bandwidth.page
+++ b/networking/bandwidth.page
@@ -17,11 +17,11 @@ users in the same region, so enforcing or charging for total throughput doesn't
 seem to make much sense?
 
 
-Rate        Latency     Ops/hour    1 mil ops   100 mil ops
----------   ---------   ---------   ----------- ------------
-1/sec       1 sec       3.6k        11.6 days   3.17 years
-20/sec      50 ms       72k         13.8 hours  57.5 days
-100/sec     10 ms       360k        2.7 hours   11.25 days
-250/sec     4 ms        0.9mil      1.1 hours   4.58 days
-1k/sec      1 ms        3.6mil      16.6 min    27.6 hours
+Rate        Latency     ops/hour    ops/day     1 mil ops   100 mil ops
+---------   ---------   ---------   ---------   ----------- ------------
+1/sec       1 sec       3.6k        86.4k       11.6 days   3.17 years
+20/sec      50 ms       72k         1.7mil      13.8 hours  57.5 days
+100/sec     10 ms       360k        8.6mil      2.7 hours   11.25 days
+250/sec     4 ms        0.9mil      21.6mil     1.1 hours   4.58 days
+1k/sec      1 ms        3.6mil      86.4mil     16.6 min    27.6 hours
 
diff --git a/software/bash.page b/software/bash.page
index 0d5325b..5d82fdc 100644
--- a/software/bash.page
+++ b/software/bash.page
@@ -37,6 +37,8 @@ Note that `join`, `grep`, and others sometimes exit non-zero return codes on
 purpose (eg, pipe input closed or found no matches, as expected), which makes
 life difficult. Sometimes `|| true` is enough to get around this.
 
+More on this: <http://redsymbol.net/articles/unofficial-bash-strict-mode/>
+
 ## General Style
 
 Google has a style guide: https://google.github.io/styleguide/shell.xml
diff --git a/software/postgres.page b/software/postgres.page
index bdb01d8..7d05074 100644
--- a/software/postgres.page
+++ b/software/postgres.page
@@ -15,3 +15,20 @@ Basic status from the psql command line:
 
     \l+     (list databases)
     \dt+    (describe table)
+
+## Temporary Databases in Tests
+
+Try `pg_tmp`, though it may not work with high concurrency (due to internal
+postgres locking), and might need special configuration in CI environments.
+
+Debian also ships with `pg_virtualenv` for temporary databases.
+
+---------
+
+More refs/links:
+
+- https://stackoverflow.com/questions/9604723/alternate-output-format-for-psql
+- https://brandur.org/postgres-connections
+- https://www.citusdata.com/blog/2017/07/16/customizing-my-postgres-shell-using-psqlrc/
+- `bind "^R" em-inc-search-prev`
+
diff --git a/software/python.page b/software/python.page
index 0fd2531..0fc3c8d 100644
--- a/software/python.page
+++ b/software/python.page
@@ -205,3 +205,24 @@ and create a `pytest.ini` like:
     norecursedirs = .svn _build tmp*
 
 Need to mock? <https://blog.fugue.co/2016-02-11-python-mocking-101.html>
+
+Debugging Memory Usage
+------------------------
+
+Most helpful tools I found were `psutil` and `pympler` (both need to be
+installed).
+
+    import os, psutil
+    process = psutil.Process(os.getpid())
+    print(process.memory_info().rss)
+    # ... do some stuff ...
+    print(process.memory_info().rss)
+
+and
+
+    from pympler import tracker
+    tr = tracker.SummaryTracker()
+    tr.print_diff()
+
+    # ... do some stuff ...
+    tr.print_diff()
diff --git a/software/rsyslog.page b/software/rsyslog.page
new file mode 100644
index 0000000..b55c127
--- /dev/null
+++ b/software/rsyslog.page
@@ -0,0 +1,20 @@
+
+Put these in files like `/etc/rsyslog.d/40-grobid-skip.conf`.
+
+Filter out lines from a given program:
+
+    :programname, isequal, "GROBID" ~
+
+Or matching a pattern:
+
+    :msg, startswith, "inserting line to HBase: sha1:"
+    :msg, contains, " INFO GET http"
+
+For list of operations ("isequal", "regex", "contains"):
+
+    https://www.rsyslog.com/doc/v8-stable/configuration/filters.html
+
+For list of properties (":msg", ":programname"):
+
+    https://www.rsyslog.com/doc/v8-stable/configuration/properties.html
+
diff --git a/software/rust.page b/software/rust.page
index df99de4..dc900c8 100644
--- a/software/rust.page
+++ b/software/rust.page
@@ -3,9 +3,10 @@ Rust
 
 ## Resources
 
-- [http://xion.io/post/code/rust-iter-patterns.html]()
-- [https://deterministic.space/rust-cli-tips.html]()
-- [https://manishearth.github.io/blog/2018/01/10/whats-tokio-and-async-io-all-about/]()
+- <http://xion.io/post/code/rust-iter-patterns.html>
+- <https://deterministic.space/rust-cli-tips.htm>
+- <https://manishearth.github.io/blog/2018/01/10/whats-tokio-and-async-io-all-about/>
+- <https://saghm.github.io/five-rust-things/>
 
 Optimization: use `RUSTFLAGS="-C target-cpu=native"` to take advantage of CPU
 special features.
@@ -22,3 +23,113 @@ Run tests with stdout output:
 
 To run tests with logging enabled (eg, with `env_logger`), make sure you add
 `env_logger::init()` to the test function itself.
+
+
+## map() and Result Ergonomics
+
+`.collect()` has some magical features! In addition to turning an iterator of
+`Item` into `Vec<Item>`, it will turn an iterator of `Result<Item>` into
+`Result<Vec<Item>>`. This makes it really useful for the end of functions.
+
+This is particularly useful for resolving some categories of "error handling in
+map closures": you can use `?` in the map closure as long as you wrap the happy
+path with `Ok()` and call collect on the outside. Eg:
+
+    let list: Vec<Item> = junk
+        .iter()
+        .map(|thing| Ok(Item {
+            a: thing.a,
+            b: fixup(thing.widget)?,
+        }))
+        .collect::Result<Vec<Item>>()?;
+
+What about when `map` over an `Option`? Eg:
+
+    let toy = Shiny {
+        a: 123,
+        b: component.map(|v| paint(v).expect("paint to succeed"),
+    };
+
+Should use match in this case:
+
+    let toy = Shiny {
+        a: 123,
+        b: match component {
+            None => None,
+            Some(v) => Some(paint(v)?),
+        },
+    };
+
+## 2020-05-17 Reading
+
+While working on fatcat-cli tool, checked the The Rust Programming Language
+book to read about trait objects and the `dyn` keyword, which I had ignored
+previously.
+
+They seem like they could be used in a few places in fatcat-server rust code.
+We don't particularly care about per-function-call performance there, and most
+entities are already allocated on the heap.
+
+Other small syntax and thing learned:
+
+Can copy a struct while only updating specific fields with ".." syntax. Might
+use this in fatcat-cli for update mutation.
+
+This is the cleanest example of using ErrorKind that I have seen:
+
+    let f = match f {
+        Ok(file) => file,
+        Err(error) => match error.kind() {
+            ErrorKind::NotFound => match File::create("hello.txt") {
+                Ok(fc) => fc,
+                Err(e) => panic!("Problem creating the file: {:?}", e),
+            },
+            other_error => {
+                panic!("Problem opening the file: {:?}", other_error)
+            }
+        },
+    };
+
+I didn't realize that test code may get compiled into non-test binaries unless
+annotated with `#[cfg(test)]`. You are supposed to create a sub-module within
+each `src/` file with unittests, like:
+
+    #[cfg(test)]
+    mod tests {
+        use super::*;
+
+        #[test]
+        fn it_works() {
+            assert_eq!(2 + 2, 4);
+        }
+    }
+This doesn't apply to `tests/` directory, which is for integration tests.
+
+The common pattern for binary crates (vs. library crates) is to have `main.rs`
+and `lib.rs`, with any code that needs to be tested in `lib.rs` (aka, all the
+actual logic).
+
+I think I knew `eprintln!()` (for stderr) vs. `println!()` (for stdout), but
+good to remember.
+
+There is a description of how to avoid memory leaks with reference counting
+using "weak" `Rc` references. Probably worth reading the [entire chapter on smart pointers](https://doc.rust-lang.org/book/ch15-06-reference-cycles.html#preventing-reference-cycles-turning-an-rct-into-a-weakt)
+(including Box, Rc, RefCell) again.
+
+For the `Sized` trait, and `Sized` trait alone, can specify an ambiguous trait
+constraint with `?` to indicate "may or may not be Sized", which doesn't really
+mean anything but does explicitly allow generic functions over non-sized traits
+like:
+
+    fn my_generic_func<T: ?Sized>(t: &T) {
+        // --snip--
+    }
+
+A trait can depend on another trait. For example, a PrettyPrint trait could
+rely on Display (and impl functions could call functions from Display). This is
+done on the trait definition line. Such a trait is called a "supertrait".
+
+Implementing Deref on a wrapper type allows transparent access to all the trait
+methods on the interior object.
+
+Also, a new longer post on error handling: <https://nick.groenen.me/posts/rust-error-handling/>