From 8c4e441050c3f4cce678bed4d3f112ba26800cb3 Mon Sep 17 00:00:00 2001 From: bryan newbold Date: Tue, 29 Jul 2008 03:36:07 -0400 Subject: made software dir, added pynfs --- MOSS | 98 ------------------------------ bash | 6 -- freebsd-packages | 46 -------------- matlab | 39 ------------ sagemath | 23 ------- software/MOSS | 98 ++++++++++++++++++++++++++++++ software/bash | 6 ++ software/freebsd-packages | 46 ++++++++++++++ software/matlab | 39 ++++++++++++ software/pynsfs | 150 ++++++++++++++++++++++++++++++++++++++++++++++ software/sagemath | 23 +++++++ 11 files changed, 362 insertions(+), 212 deletions(-) delete mode 100644 MOSS delete mode 100644 bash delete mode 100644 freebsd-packages delete mode 100644 matlab delete mode 100644 sagemath create mode 100644 software/MOSS create mode 100644 software/bash create mode 100644 software/freebsd-packages create mode 100644 software/matlab create mode 100644 software/pynsfs create mode 100644 software/sagemath diff --git a/MOSS b/MOSS deleted file mode 100644 index 4de6d23..0000000 --- a/MOSS +++ /dev/null @@ -1,98 +0,0 @@ -==== -MOSS -==== --------------------------------- -Many User Operating System Stuff --------------------------------- - -:Author: Bryan Newbold - -Moss is a vague concept I have for an operating-system-like-system that -attempts to realize some of the promises of distributed universal computing -and user management, continuation-based serializable programming, persistent -data accessibility, file and process versioning, central foreign function -management, and code/content distribution. It'll be easy! - -.. topic:: Implementation - - Moss would probably start as "stuff": middleware, userland applications, - utilities, and native shell and GUI interfaces. It could also be a - separate hosted virtual machine, a monolithic application, a kernel - extension, or ultimately run alone over a high performance shim host OS. - - Distribution would be self hosting and viral: users would replicate a copy - of the system from a friend instead of from a central server, patches - and applications would be distributed word-of-mouth, and trust networks - would form naturally via this distribution. Customization and feature sets - would be passed on, which makes it likely that a user would receive a - system already tweaked for their own needs and computing knowledge level. - - *Existing Projects:* Inferno, Xen, vmware, Java, GNU/* - -.. topic:: Universal, distributed file system - - The core of the system would be a universally accessible identity-based - operating system. Some authoritive domain would probably be required, but - public identity brokers would allow anonymous identities. "Strong - Cryptography" is a goal, allowing a user's content to be hosted/cached - on third party machines in an encrypted form. The real challenge of course - is a flexible crypto system than can be transitioned or upgraded if a flaw - is discovered without total data loss. - - My dream path would look something like:: - - /net/user@some.domain.tld/media/ledzep/tangerine.mp3 - - From the application end there would be no concept of "local" or "remote" - files to a particular machine, though perhaps some feedback on access time. - So, for instance, once tokens/authentication is handled, user utilities - like ``mv`` or ``cat`` could be applied, instead of ``scp`` or ``rcat``. - - Versioning, write locks, etc would have to be considered. - - *Existing projects:* OpenAFS, freeNet, ssh, kerberos, git - -.. topic:: Serializable Programs - - The state/continuation/environment of a running program or chain of - programs should be a "first level object": a bundle of data like any other - that can be transmitted, copied, and stored away for later. A user should - be able to drag an entire application running on a desktop computer - onto their laptop when then need to travel, or from laptop to workstation - if then need additional computing power. Distributed computing could be - implemented by bundling up applets that are shot off to a cluster or - higher performance computer for processing, and the result state of the - program would simply be bundled back to the requesting client. Such bundles - wouldn't be very large: data would be stored on the distributed filesystem, - which appears identical (*exactly?*) to every node on the network. - - Properly written, such a serializable system could also lead to performance - and power consumption savings by swapping idle programs and processes to - disk, or let low-usage nodes shift their processes off to other nodes - and power down. - - *Existing Projects:* Lisp, Stackless - -.. topic:: Foreign Function Management - - It would be nice to see a move away from the library model for shared - code to a more flexible/dynamic foreign function interface that would - allow any appropriate code to announce its availability to other - applications regardless of version, platform, coding language, etc. - This would be a high-level feature, not intended to replace kernel level - operations (read/write) but to make package/library management easier - (it doesn't matter if an image conversion function is coming from a video - editing package or libpng as long as it reads a raw array and returns - a binary stream). - - There's room for dynamic optimization here: if program - realizes it's native string manipulation library sucks for 5meg+ datasets - it could look through the library and see if there's a better one. - - *And,* this too could be distributed, allowing super easy access to - distributed computing resources; underutilized nodes could make their - functions available to nearby nodes, or a machine with tons of matrix - crunching silicon (eg high end video cards) could swap work units - with a machine with a dedicated crypto chip or 64bit+ processor. - - *Existing Projects:* Script-fu from the Gimp diff --git a/bash b/bash deleted file mode 100644 index e3402fb..0000000 --- a/bash +++ /dev/null @@ -1,6 +0,0 @@ -================== -BASH -================== - -Job Control - The syntax for "job number" or "JOBSPEC" (when using ``kill`` or similar) is ``%4`` or ``%5``. diff --git a/freebsd-packages b/freebsd-packages deleted file mode 100644 index 642a615..0000000 --- a/freebsd-packages +++ /dev/null @@ -1,46 +0,0 @@ -======================== -FreeBSD Packages -======================== - -These are the packages I typically install for a workstation; this is after selecting an "X windows kernel developer" base system with ``sshd``, linux binary support, and NTP configured:: - - xmms-wma - xmms-flac - xmms-faad2 - xmms - vim - emacs - mplayer - pork - ctorrent - teTeX - ipython - mit-scheme - py-numpy - py25-matplotlib - py25-scipy - bash - lighttpd - firefox - apache - fluxbox - xfce - git - subversion - pine - alpine - screen - python - vlc - gimp - xzgc - lyx - wget - -The following are added by hand from ports:: - - vlc - java - gtkchtheme - e2fsprogs - diff --git a/matlab b/matlab deleted file mode 100644 index 20e7110..0000000 --- a/matlab +++ /dev/null @@ -1,39 +0,0 @@ -================== -Matlab -================== - -.. warning:: - Stub, just getting started - -`cyclismo.org `_ has a good tutorial. - -Matrix Syntax ------------------- -A = [1,2,3,4] - creates a new matrix -A(1) - selects the first element of a matrix, **zero-indexed** -B = [ [1,2,3], [4,5,6], [7,8,9] ] - creates a one dimensional matrix - -:: - - B = [ [1,2,3], - [4,5,6], - [7,8,9] ] - - creates a two dimensional matrix - -Matrix Operations ------------------- -``inv`` - inverts a square matrix - -Meta-Commands ------------------- -Placing a semicolon at the end of a line prevents the output from being printed to screen. - -``whos`` - Displays variables and their size -``format long``/``format short`` - Selects either 4 or 14 decimal place display diff --git a/sagemath b/sagemath deleted file mode 100644 index f01883f..0000000 --- a/sagemath +++ /dev/null @@ -1,23 +0,0 @@ -================== -SAGE Math Package -================== - -.. warning:: Under progress! - -This is a reference for the `sage `_ math package. - -Web Notebooks ------------------------------ -To start run ``notebook()`` from an interactive shell; passing a directory as an argument will store files there. Also try``sage_wiki()``! - -To get help on a ``cmd`` type ``cmd?`` in an cell and press the Escape key. - -Interactive Shell ------------------------------ -Instead of the usual python ``**`` you can use ``^`` like the rest of the world, at least from the notebook and shell; use the old operator in scripts? - -Prefacing with ``time`` gives processor time statistics; you can wrap a series of commands with ``t = cputime()`` and ``cputime(t)`` to get the sum of time statistics; ``walltime`` works similarly. - -The general trick for saving text representations of objects is to open a file dscriptor and write the ``str`` of the object to it. - -``save_session`` and ``load_session`` save all objects in the session to a ``.sobj`` file. diff --git a/software/MOSS b/software/MOSS new file mode 100644 index 0000000..4de6d23 --- /dev/null +++ b/software/MOSS @@ -0,0 +1,98 @@ +==== +MOSS +==== +-------------------------------- +Many User Operating System Stuff +-------------------------------- + +:Author: Bryan Newbold + +Moss is a vague concept I have for an operating-system-like-system that +attempts to realize some of the promises of distributed universal computing +and user management, continuation-based serializable programming, persistent +data accessibility, file and process versioning, central foreign function +management, and code/content distribution. It'll be easy! + +.. topic:: Implementation + + Moss would probably start as "stuff": middleware, userland applications, + utilities, and native shell and GUI interfaces. It could also be a + separate hosted virtual machine, a monolithic application, a kernel + extension, or ultimately run alone over a high performance shim host OS. + + Distribution would be self hosting and viral: users would replicate a copy + of the system from a friend instead of from a central server, patches + and applications would be distributed word-of-mouth, and trust networks + would form naturally via this distribution. Customization and feature sets + would be passed on, which makes it likely that a user would receive a + system already tweaked for their own needs and computing knowledge level. + + *Existing Projects:* Inferno, Xen, vmware, Java, GNU/* + +.. topic:: Universal, distributed file system + + The core of the system would be a universally accessible identity-based + operating system. Some authoritive domain would probably be required, but + public identity brokers would allow anonymous identities. "Strong + Cryptography" is a goal, allowing a user's content to be hosted/cached + on third party machines in an encrypted form. The real challenge of course + is a flexible crypto system than can be transitioned or upgraded if a flaw + is discovered without total data loss. + + My dream path would look something like:: + + /net/user@some.domain.tld/media/ledzep/tangerine.mp3 + + From the application end there would be no concept of "local" or "remote" + files to a particular machine, though perhaps some feedback on access time. + So, for instance, once tokens/authentication is handled, user utilities + like ``mv`` or ``cat`` could be applied, instead of ``scp`` or ``rcat``. + + Versioning, write locks, etc would have to be considered. + + *Existing projects:* OpenAFS, freeNet, ssh, kerberos, git + +.. topic:: Serializable Programs + + The state/continuation/environment of a running program or chain of + programs should be a "first level object": a bundle of data like any other + that can be transmitted, copied, and stored away for later. A user should + be able to drag an entire application running on a desktop computer + onto their laptop when then need to travel, or from laptop to workstation + if then need additional computing power. Distributed computing could be + implemented by bundling up applets that are shot off to a cluster or + higher performance computer for processing, and the result state of the + program would simply be bundled back to the requesting client. Such bundles + wouldn't be very large: data would be stored on the distributed filesystem, + which appears identical (*exactly?*) to every node on the network. + + Properly written, such a serializable system could also lead to performance + and power consumption savings by swapping idle programs and processes to + disk, or let low-usage nodes shift their processes off to other nodes + and power down. + + *Existing Projects:* Lisp, Stackless + +.. topic:: Foreign Function Management + + It would be nice to see a move away from the library model for shared + code to a more flexible/dynamic foreign function interface that would + allow any appropriate code to announce its availability to other + applications regardless of version, platform, coding language, etc. + This would be a high-level feature, not intended to replace kernel level + operations (read/write) but to make package/library management easier + (it doesn't matter if an image conversion function is coming from a video + editing package or libpng as long as it reads a raw array and returns + a binary stream). + + There's room for dynamic optimization here: if program + realizes it's native string manipulation library sucks for 5meg+ datasets + it could look through the library and see if there's a better one. + + *And,* this too could be distributed, allowing super easy access to + distributed computing resources; underutilized nodes could make their + functions available to nearby nodes, or a machine with tons of matrix + crunching silicon (eg high end video cards) could swap work units + with a machine with a dedicated crypto chip or 64bit+ processor. + + *Existing Projects:* Script-fu from the Gimp diff --git a/software/bash b/software/bash new file mode 100644 index 0000000..e3402fb --- /dev/null +++ b/software/bash @@ -0,0 +1,6 @@ +================== +BASH +================== + +Job Control + The syntax for "job number" or "JOBSPEC" (when using ``kill`` or similar) is ``%4`` or ``%5``. diff --git a/software/freebsd-packages b/software/freebsd-packages new file mode 100644 index 0000000..642a615 --- /dev/null +++ b/software/freebsd-packages @@ -0,0 +1,46 @@ +======================== +FreeBSD Packages +======================== + +These are the packages I typically install for a workstation; this is after selecting an "X windows kernel developer" base system with ``sshd``, linux binary support, and NTP configured:: + + xmms-wma + xmms-flac + xmms-faad2 + xmms + vim + emacs + mplayer + pork + ctorrent + teTeX + ipython + mit-scheme + py-numpy + py25-matplotlib + py25-scipy + bash + lighttpd + firefox + apache + fluxbox + xfce + git + subversion + pine + alpine + screen + python + vlc + gimp + xzgc + lyx + wget + +The following are added by hand from ports:: + + vlc + java + gtkchtheme + e2fsprogs + diff --git a/software/matlab b/software/matlab new file mode 100644 index 0000000..20e7110 --- /dev/null +++ b/software/matlab @@ -0,0 +1,39 @@ +================== +Matlab +================== + +.. warning:: + Stub, just getting started + +`cyclismo.org `_ has a good tutorial. + +Matrix Syntax +------------------ +A = [1,2,3,4] + creates a new matrix +A(1) + selects the first element of a matrix, **zero-indexed** +B = [ [1,2,3], [4,5,6], [7,8,9] ] + creates a one dimensional matrix + +:: + + B = [ [1,2,3], + [4,5,6], + [7,8,9] ] + + creates a two dimensional matrix + +Matrix Operations +------------------ +``inv`` + inverts a square matrix + +Meta-Commands +------------------ +Placing a semicolon at the end of a line prevents the output from being printed to screen. + +``whos`` + Displays variables and their size +``format long``/``format short`` + Selects either 4 or 14 decimal place display diff --git a/software/pynsfs b/software/pynsfs new file mode 100644 index 0000000..7c1951d --- /dev/null +++ b/software/pynsfs @@ -0,0 +1,150 @@ +============================================= +Python Namespace File System +============================================= + +:Author: Bryan Newbold +:Date: July 2008 +:URL: http://git.bryannewbold.com?p=pynsfs;a=summary + +.. contents:: + +This is an experimental implementation of a python namespace as a virtual +file system, implemented using the python bindings for fusefs on unix. + +At this point only the global namespace of the filesystem instance itself is +accessible, and only limited access is available to that. If anything goes +wrong it can cause a hard lock and system crash; don't leave it running and +don't try to write anything substantial. + +The source code can be checked out from the above URL. + +Features +--------------- +An example session goes like:: + + snark# ls /mnt/python/ + + snark# python nsfs.py /mnt/python/ + *** init complete + let's go! + + snark# cd /mnt/python/ + + snark# ls -l + total 6 + -r--r--r-- 1 root wheel 30 Dec 31 1969 DefaultStat + -r--r--r-- 1 root wheel 33 Dec 31 1969 DefaultStatVfs + -r--r--r-- 1 root wheel 19 Dec 31 1969 Fuse + -r--r--r-- 1 root wheel 30 Dec 31 1969 NamespaceFS + -r--r--r-- 1 root wheel 27 Dec 31 1969 NsfsFile + -r--r--r-- 1 root wheel 17 Dec 31 1969 StringIO + drwxr-xr-x 2 root wheel 0 Dec 31 1969 __builtins__ + -r--r--r-- 1 root wheel 4 Dec 31 1969 __doc__ + -r--r--r-- 1 root wheel 7 Dec 31 1969 __file__ + -r--r--r-- 1 root wheel 8 Dec 31 1969 __name__ + drwxr-xr-x 2 root wheel 0 Dec 31 1969 _find_fuse_parts + drwxr-xr-x 2 root wheel 0 Dec 31 1969 errno + drwxr-xr-x 2 root wheel 0 Dec 31 1969 fuse + -r--r--r-- 1 root wheel 29 Dec 31 1969 main + drwxr-xr-x 2 root wheel 0 Dec 31 1969 os + drwxr-xr-x 2 root wheel 0 Dec 31 1969 stat + -r--r--r-- 1 root wheel 24 Dec 31 1969 time + + snark# ls stat/ + ST_ATIME S_ENFMT S_IMODE S_ISDIR S_IWRITE + ST_CTIME S_IEXEC S_IREAD S_ISFIFO S_IWUSR + ST_DEV S_IFBLK S_IRGRP S_ISGID S_IXGRP + ST_GID S_IFCHR S_IROTH S_ISLNK S_IXOTH + ST_INO S_IFDIR S_IRUSR S_ISREG S_IXUSR + ST_MODE S_IFIFO S_IRWXG S_ISSOCK __builtins__ + ST_MTIME S_IFLNK S_IRWXO S_ISUID __doc__ + ST_NLINK S_IFMT S_IRWXU S_ISVTX __file__ + ST_SIZE S_IFREG S_ISBLK S_IWGRP __name__ + ST_UID S_IFSOCK S_ISCHR S_IWOTH + + snark# cat time + + + snark# echo " Hello Python" >> os/__name__ + + snark# cat os/__name__ + os Hello Python + + snark# rm time + + snark# ls + DefaultStat __builtins__ fuse + DefaultStatVfs __doc__ main + Fuse __file__ os + NamespaceFS __name__ stat + NsfsFile _find_fuse_parts + StringIO errno + + snark# ln __nameofos__ os/__name__ + ln: __nameofos__: No such file or directory + + snark# ln os/__name__ __nameofos__ + + snark# cat __nameofos__ + os Hello Python + + snark# cd .. + + snark# umount /mnt/python/ + +The root directory is the global namespace; modules are available as +subdirectories and all other objects are accessible as files. Strings can +be appended to; all other "files" are readonly and return their string +representation. "Files" can be deleted; "directories" can not. Hard links to +files are possible. + +Requirements +--------------- + +This was developed and has only been tested on FreeBSD 7.0 with Python 2.5.1 +version 0.2 of the python-fuse bindings. + +* `FUSE Filesystem`__ installed and configured on a compatible operating system +* Recent version of Python__ +* `FUSE Python Bindings`__ + +Once everything is installed and configured, the script can just be run +as root; no installation necessary. + +__ http://fuse.sourceforge.net/ +__ http://python.org +__ http://fuse.sourceforge.net/wiki/index.php/FusePython + +Usage +-------------- +Try ``# python nsfs.py -h`` to get a list of general FUSE options. A simple +mount command would look like:: + + # python nsfs.py /mnt/python + +To unmount:: + + # umount /mnt/python + +I like to run in foreground, single threaded, with debug info:: + + # python nsfs.py -dsf /mnt/python + +Really only basic command line tools work because they don't check for many +attributes; try ``echo "blah" >> file``, ``cat file``, ``df -h``, ``ls -l`` etc. + +TODO +----------------------- +Functions taking simple arguments (or no arguments) could be implemented as +character devices; ``function_name.doc``, ``function_name.repr``, etc could +give meta-information. + +File of the form ``__thing__`` should be renamed ``.thing``. + +New files should be allowed and saved as strings. + +CHANGELOG +----------------------- + +July 28, 2008 + First implementation diff --git a/software/sagemath b/software/sagemath new file mode 100644 index 0000000..f01883f --- /dev/null +++ b/software/sagemath @@ -0,0 +1,23 @@ +================== +SAGE Math Package +================== + +.. warning:: Under progress! + +This is a reference for the `sage `_ math package. + +Web Notebooks +----------------------------- +To start run ``notebook()`` from an interactive shell; passing a directory as an argument will store files there. Also try``sage_wiki()``! + +To get help on a ``cmd`` type ``cmd?`` in an cell and press the Escape key. + +Interactive Shell +----------------------------- +Instead of the usual python ``**`` you can use ``^`` like the rest of the world, at least from the notebook and shell; use the old operator in scripts? + +Prefacing with ``time`` gives processor time statistics; you can wrap a series of commands with ``t = cputime()`` and ``cputime(t)`` to get the sum of time statistics; ``walltime`` works similarly. + +The general trick for saving text representations of objects is to open a file dscriptor and write the ``str`` of the object to it. + +``save_session`` and ``load_session`` save all objects in the session to a ``.sobj`` file. -- cgit v1.2.3