--- format: rst toc: no ... ====================== Python ====================== iPython tricks -------------- Use the ``cpaste`` command to copy in blocks of raw python code, even if the indenting is a little weird. Style ------- `Python PEP-008 `_: Style Guide for Python Code `pylint `_, a Python syntax checker. Very verbose, use pylint -E (errors only) or at least ``pylint -r no`` (no report). Eg, ``pylint -r no file.py -d W0614 -d C -d R``. For docstring documentation, refer to `PEP-257 `_ and the `Sphinx documentation `_; specifically, document script functionality in a top level (above imports, below any hashbang) docstring. Use leading ``#:`` style comments to document important non-object/non-function element definitions (eg, static variables) in a way that will get pulled out into Sphinx. Use "Google-style" function argument/return documentation instead of "Sphinx style". For example:: def public_fn_with_googley_docstring(name, state=None): """This function does something. Args: name (str): The name to use. Kwargs: state (bool): Current state to be in. Returns: int. The return code:: 0 -- Success! 1 -- No good. 2 -- Try again. Raises: AttributeError, KeyError A really great idea. A way you might use me is >>> print public_fn_with_googley_docstring(name='foo', state=None) 0 BTW, this always returns 0. **NEVER** use with :class:`MyPublicClass`. """ return 0 autopep8 is a tool to automatically pep8-ify a file. Use it like:: sudo pip install autopep8 autopep8 --in-place --select=W293,W191,W291 *.py pep8radius is sort of similar, but only applies to code that you are going to commit (using VCS info). Packaging ----------- Flask app packaging advice, including ``MANIFEST.in`` and non-PyPi dependancy advice: http://flask.pocoo.org/docs/patterns/distribute/ Use ``console_scripts`` in ``setup.py`` to install system-wide scripts: http://packages.python.org/distribute/setuptools.html#automatic-script-creation For debian packaging, use `stdeb `_ (via `stackoverflow thread `_). For notes on pip vs. setup.py dependencies: https://caremad.io/blog/setup-vs-requirement/ "Fucking" String Encoding --------------------------- (str/unicode errors in python are very prevalent and give me the rage) The ``codecs`` package has some helpers; see for example ``open(f,mode,encoding)``. ASCII ---------------- 'ord' is the function that takes a single ASCII character and returns the value number (as an int). RunSnakeRun ------------- Runsnake seems to be unmaintained... `snakeviz` is the new thing? Example session:: $ python -m cProfile -o ./dump.profile myscript.py --script-option blah $ # run to completion or Ctrl-C, then $ runsnakerun ./dump.profile # or $ snakeviz ./dump.profile nosetests ------------- NOTE: by default I now use pytest instead of nose To do minimal tests without wrapping everything in a class, import assert functions from nose.tools, eg:: from nose.tools import assert_raises, assert_equal To do interactive pdb debugging, simply:: $ nosetests --pdb # or sometimes: $ nosetests --pdb-failures pdb ------- To debug a script (interactive prompt on exception):: python -m pdb myscript.py or in-line, always at a particular point:: import pdb; pdb.set_trace() Use ipdb (debian: python-ipdb) instead of pdb to get a nicer IPython prompt. Python 3 Porting ------------------- To help port and support both Python 2.7 and 3.0+, start with an import:: from __future__ import absolute_import, division, print_function str/bytes/unicode is indeed the major porting challenge. Using bytearrays helps. Use ``b'asdf'`` style byte array definitions for most low-level constants. ``struct.unpack()`` wants doesn't allow ``bytearray()``; use ``bytes()`` instead. Make sure ``rase Exception ("Message here")`` style is used everywhere, instead of ``raise Exception, "Message here"``. There was some change in comparison between None and integers, which makes ``if args.verbose > 0`` break. Use ``if args.verbose and args.verbose > 1`` instead. Debian Default Python ---------------------- Keep in mind that this can break some system things (eg, arandr, some cups things). To make Python 3.4 the default for `python` system-wide, do something like: sudo update-alternatives --install /usr/bin/python python /usr/bin/python3.4 2 For an individual user you can also just create a `python` alias. See also: http://linuxconfig.org/how-to-change-from-default-to-alternative-python-version-on-debian-linux Nice Packages -------------- subprocess/pexpect: https://github.com/kennethreitz/delegator.py time/datetime: https://github.com/crsmithdev/arrow tsv: https://github.com/kennethreitz/tablib simple HTML scraping: https://github.com/michaelhelmick/lassie sqlite3dbm is a library to back a python dict with sqlite3 on disk pytest ----------- pytest sort of "just works" if you put test files under `./tests/`. If you want crude file imports, skipping directories, and including `test_*` functions from any python file (not just those with `test_*.py`, install `pytest-pythonpath` and create a `pytest.ini` like: [pytest] python_paths = . python_files = *.py norecursedirs = .svn _build tmp* Need to mock? Debugging Memory Usage ------------------------ Most helpful tools I found were `psutil` and `pympler` (both need to be installed). import os, psutil process = psutil.Process(os.getpid()) print(process.memory_info().rss) # ... do some stuff ... print(process.memory_info().rss) and from pympler import tracker tr = tracker.SummaryTracker() tr.print_diff() # ... do some stuff ... tr.print_diff()