--- format: rst toc: no ... ======================================= Miscellaneous Software Stuff ======================================= Log Analysis --------------- http://logstash.net/ Regular Expressions --------------------- http://code.google.com/p/semicomplete/wiki/GrokDiscovery http://txt2re.com/ Compression for Executables --------------------------- UPX: Ultimate Packer for Executables http://upx.sourceforge.net/ Webmail ------- the null webmail client http://www.nulllogic.ca/webmail/ Standard ML ----------- `The SML Standard Basis (of types) `_ Mutually recursive function using 'and' syntax: fun foo 0 = "beer car" | foo n = bar(n-1) and bar 0 = "bacon" | bar n = foo(n-1); Network programming in SML/NJ: https://github.com/tonyg/smlnj-networking/ plan9 and Inferno -------------------- "Styx-on-a-Brick": networked LEGO device http://doc.cat-v.org/inferno/4th_edition/styx-on-a-brick/ plan9 on a GuruPlug http://plan9.bell-labs.com/wiki/plan9/guruplug/index.html NoSQL ------ NoSQL Data Modeling Techniques http://highlyscalable.wordpress.com/2012/03/01/nosql-data-modeling-techniques/ PHP Problems -------------- [PHP: A fractal of bad design](http://me.veekun.com/blog/2012/04/09/php-a-fractal-of-bad-design/) [PHP Sucks but it doesn't matter](http://www.codinghorror.com/blog/2008/05/php-sucks-but-it-doesnt-matter.html) LISP Visualization ------------------ Nested boxes instead of nested parens: http://pretty-lisp.org/ Webmail ------- Squirrel mail is ok, CiderWebmail looks better. Have had problems deploying Alpine Webmail. Structured Wikis ---------------- Circa May 2012 there are a few options for structured data in wikis: *[Semantic MediaWiki](https://en.wikipedia.org/wiki/Semantic_MediaWiki)* is probably the best option, though it is unfortunately in PHP. This is a series of plugins/extensions to vanilla MediaWiki, which add editing (with or without forms), querying, visualization, import/export, etc. *MediaWiki with the [WikiDB](http://www.mediawiki.org/wiki/Extension:WikiDB)* (PHP) *DokuWiki with [data plugin](http://www.dokuwiki.org/plugin:data)* (PHP) *eLog* could be used to collaboratively edit simple data. *TWiki* has structured data features and an input form (Perl). DBPedia uses [custom extraction tools](http://wiki.dbpedia.org/Documentation?v=6hz) for MediaWiki written in Scala. dbpedialite uses just calls the Wikipedia API (Ruby). You can [search](http://www.wikimatrix.org/search.php?sid=54395) wikimatrix for wiki engines which have a "structured data" feature. Semantic Data Publishing ------------------------ [OntoWiki](http://ontowiki.net/Projects/OntoWiki/Features) [Google Refine](https://en.wikipedia.org/wiki/Google_Refine) (formerly known as Freebase Gridworks) seems pretty useful for "cleaning up" datasets. It's a Java "workstation" app with a web/http interface. USB Performance Issues ---------------------- Parallelized mass storage transfers are much slower than serialized: http://blog.dinkevich.com/why-usb-transfers-are-slow/ Inkscape Black Rectangle Box Issue ------------------------------------- For some reason Inkscape sometimes inserts a black box into .svg files which are invisible within Inkscape but visible when exported. To remove this box, edit the raw XML using the XML Editor in Inkscape (Shift-Ctrl-X), look for an svg:flowRoot element within the first layer, and delete it. Latency Numbers ----------------- From https://gist.github.com/2841832:: L1 cache reference 0.5 ns Branch mispredict 5 ns L2 cache reference 7 ns 14x L1 cache Mutex lock/unlock 25 ns Main memory reference 100 ns 20x L2 cache, 200x L1 cache Compress 1K bytes with Zippy 3,000 ns Send 1K bytes over 1 Gbps network 10,000 ns 0.01 ms Read 4K randomly from SSD* 150,000 ns 0.15 ms Read 1 MB sequentially from memory 250,000 ns 0.25 ms Round trip within same datacenter 500,000 ns 0.5 ms Read 1 MB sequentially from SSD* 1,000,000 ns 1 ms 4X memory Disk seek 10,000,000 ns 10 ms 20x datacenter roundtrip Read 1 MB sequentially from disk 20,000,000 ns 20 ms 80x memory, 20X SSD Send packet CA->Netherlands->CA 150,000,000 ns 150 ms Notes: 1 ns = 10-9 seconds 1 ms = 10-3 seconds * Assuming ~1GB/sec SSD Great Hacks ---------------- HTML as image: http://lcamtuf.coredump.cx/squirrel/ xscreensaver ------------- To lock the screen immediately, do: xscreensaver-command -lock Lua Tools ------------- http://haserl.sourceforge.net/ "Dynamic web content in 20K" Semantic Wiki Setup Notes --------------------------- There is a [](Semantic MediaWiki quick reference), as well as good documentation of the [http://www.mediawiki.org/wiki/Extension:Semantic_Forms/Quick_start_guide](Semantic Forms) plugin. [http://www.mediawiki.org/wiki/Extension:External_Data](External Data) plugin allows remote import of data via HTTP API call (cached) or file upload (eg, XML, csv). "The djb Way" -------------- [Dan Bernstein](http://cr.yp.to/djb.html) writes a lot of network daemons in a unique style. Eg, daemontools, ucspi-tcp, djbdns, qmail, and publicfile. - [The djb way](http://thedjbway.b0llix.net/readme.html) How TTYs Work ---------------- Covers ``nohup``, piping, process groups, session leaders, TTYs, PTYs, ``stty``, etc. Web Security ----------------- Capture all HTTP Traffic to File ---------------------------------- Install ``tcpflow`` and do: $ mkdir ~/full_dump; cd ~/full_dump $ sudo tcpflow -i wlan0 -b 2097152 tcp port 80 or tcp port 443 Saves up to 2MB wget -------- For recursive, I usually want something like: wget -r -l 3 To just get files, ignoring structure: wget -r -l 3 -nd QEMU with chroot ------------------- Manual way: $ sudo chroot ./rootfs-dir/ qemu-arm-static /usr/bin/env The automatic way is to copy the host's qemu-arm-static to usr/bin in the chroot, then just run the command and qemu-arm-static will be used automagically: $ sudo chroot ./rootfs-dir/ /usr/bin/env Starting an SSL CA ---------------------- "[cryptography] How much does it cost to start a root CA ?" Jon Callas , Sat Jan 5 14:42:31 EST 2013 http://lists.randombit.net/pipermail/cryptography/2013-January/003601.html Answer: $250k over 14 months, 40% hardware, 60% people