View of /Contextual/Contextual.txt

Parent Directory

Revision Log

Bug fixes for state initialization and registry wildcard lookups.

===================================================================
Creating Ultralight Frameworks with "Contextual" (``peak.context``)
===================================================================

-----------------
Developer's Guide
-----------------

The ``Contextual`` library provides a safe replacement for the use of global
(or thread-local) variables, making it easy to create non-invasive frameworks
that are still highly testable and configurable.  Code based on ``Contextual``
is also safe for use in event-driven, coroutine, or "co-operative" multitasking
systems such as Twisted and ``peak.events``, because an entire thread's context
can be easily saved and restored when switching between different logical
threads, tasks, or coroutines.

Although ``Contextual`` works with Python versions 2.3 and up, the examples in
this guide are written using Python 2.5 and the ``with`` statement for ease of
reading.  If you're using an older version of, you'll need to consult the
section below on `Support for Python 2.3/2.4`_ before you try using any of the
examples that use the ``with`` statement.

Also, under Python 2.5, you'll need to do a ``__future__`` import at the top
of every module that uses the ``with`` statement, and of course you'll need
to import the ``Contextual`` library too:

    >>> from __future__ import with_statement
    >>> from peak import context


Overview
========

Global variables (including so-called "singletons") are attractive to use,
because they're so *convenient* to use.  But they're bug-prone for two reasons:

1. They're global (i.e., shared by the entire program), and

2. They're variable (i.e., can be changed at any time)!

The ``Contextual`` library lets you replace "global variables" with "contextual
values" (and "singletons" with "services").  These tools are almost as easy to
use as their "global" counterparts, but are less bug-prone becuase they're
neither "global" nor "variable"!

First, while global variables and singletons are stored in modules, contextual
values and services are stored in a "context".  By default, each thread has its
own current context, so you don't have to explicitly keep track of what context
you're in.

This gives you the convenience of global variables (i.e., you don't need to
pass them around everywhere), but without necessarily sharing the values
throughout the entire program.  Thus, they are not truly "global".

Second, contextual values and services are not "variable" either, because they
can only be set *once* within a given context (rather like variables in a
"pure functional" language).  This makes them 100% thread-safe, and free of
"race conditions" even in non-threaded code.

To "change" a value, you must create a new context (e.g. using the Python 2.5
``with:`` statement), and then assign the new value in the new context.  Then,
when the context is exited, the previous value is restored.


Safe Configuration Using Settings
=================================

The simplest type of "contextual" value is a ``context.setting``.  Settings
consist of a default "input" value, and a function for transforming input
values to the form needed when using the setting.  (For example, you might have
a setting whose output value is numeric, but which allows string inputs.)  In
the simplest case, however, a setting function can simply return its input
unchanged.

Settings are designed to be a safe replacement for module-level configuration
variables.  Instead of one program-wide value for a setting, individual code
paths can safely alter the current configuration temporarily, without affecting
any other code (including code in other threads) that needs to change or use
those settings.  Settings are also useful as replacements for rarely-changed
parameters that must be passed through a lot of function or method calls.

Let's look at two simple settings::

    >>> @context.setting
    ... def speed(value=16):
    ...     """Speed of duplicating CDs"""
    ...     return float(value)

    >>> @context.setting
    ... def resolution(value=300):
    ...     """DPI for printing CD labels"""
    ...     return int(value)

Settings look, for all intents and purposes, like normal functions, but they
are actually specialized objects that wrap the supplied function.

When called, a setting returns its "current" value.  By default, this will be
the result of applying the original function to its default input value::

    >>> speed()     # returns the default value
    16.0
    >>> resolution()
    300

However, settings can be "changed" in a controlled fashion, by entering a new
context, and using the ``<<=`` operator to set a new input value.  For
example::

    >>> with context.new():  # enter a new child context
    ...     speed <<= 48     # set speed's input to 48 in the current context
    ...     print speed()
    ...     print resolution()  # inherits the default from the outer context
    48.0
    300

    >>> speed() # outside the block, goes back to the old value
    16.0


Within a given context, however, a setting is guaranteed to only have **one**
readable value.  You can set it to the *same* (i.e. equal) input value as many
times as you like::

    >>> speed <<= 16    # setting to the existing value, ok
    >>> speed <<= 16    # still ok

But you cannot *change* the value, once it has been read in a given context::
    
    >>> speed <<= 48    # changing the value, not ok!
    Traceback (most recent call last):
      ...
    InputConflict: (speed, 16, 48)

As you can see, we get an ``InputConflict`` error showing that an input value
of ``16`` for ``speed`` has already been read in the current context, and an
attempt was made to change it to ``48``.

Remember, settings are not the same thing as "variables": they can only "vary"
in a controlled way.  If you want to "change" a setting, you must do so in
a context where that setting has not yet been used.  As you'll see in this next
example, it's the act of *using* a setting that makes it unchangeable in that
context::

    >>> with context.new():
    ...     speed <<= 77    # change as many times as we like, because the
    ...     speed <<= 99    # input hasn't been *used* yet in this context
    ...     speed <<= 66
    ...     print speed()   # but now it can't be changed ever again!
    ...
    ...     try:
    ...         speed <<= 8
    ...     except context.InputConflict:
    ...         print "Caught a conflict"
    ...
    ...     with context.new():
    ...         speed <<= 99    # but we can change it in this *new* context!
    ...         speed <<= 54    # as many times as we like
    ...         print speed()   # ...until it's read, of course
    ...
    ...     print speed()   # back to 66, as we're back in this context
    ... 
    ...     with context.new():
    ...         print speed()   # and of course it's 66 here too, inherited
    ...                         # from our containing context
    ...         
    66.0
    Caught a conflict
    54.0
    66.0
    66.0
    
This "write many until read once" behavior makes it impossible to have
setting initialization bugs where one piece of code reads one value for a
setting, and then another piece of code changes the setting to something else.


Creating and Using Services
===========================

Here's a simple "counter" service implementation::

    >>> class Counter(context.Service):
    ...     value = 0
    ...     def inc(self):
    ...         self.value += 1

You can get the current instance of the service using its ``get()``
classmethod::

    >>> count = Counter.get()   # get the current instance
    >>> count.value
    0
    >>> count.inc()
    >>> count.value
    1

By default, each thread will have its own unique instance, that's created upon
first use, and is cached thereafter::

    >>> count is Counter.get()  # still using the same one
    True

    >>> def run_in_another_thread():
    ...     c2 = Counter.get()
    ...     print c2 is count           # this will be different
    ...     print c2 is Counter.get()   # but the same as long as thread runs

    >>> import threading
    >>> t = threading.Thread(target=run_in_another_thread)

    >>> t.start(); t.join() # run and wait for it to finish
    False
    True

You can replace the current instance with another instance, by using the
``with:`` statement (or a special function in older versions of Python; see
the section below on `Support for Python 2.3/2.4`_ for more info)::

    >>> with Counter.new():
    ...     print count is Counter.get()
    False

    >>> count is Counter.get()  # old value is restored
    True


Easy Access Shortcuts
---------------------

Although up to this point we've been using ``Counter.get()`` to get the current
``Counter`` instance, note that we can also just use class attributes and
methods of ``Counter``, and they will automatically be redirected to the
corresponding attributes and methods of the current instance::

    >>> Counter.value
    1
    >>> Counter.inc()
    >>> Counter.value
    2
    >>> Counter.value = 42
    >>> Counter.get().value
    42

Any attribute that is defined in the service class that is *not* a
staticmethod, classmethod, or language-defined class attribute (like
``__new__`` and ``__slots__``) is automatically set up to redirect to the
instances, when retrieved from the class.

In other words, ``Counter.inc`` and ``Counter.value`` are just
shorter ways to spell ``Counter.get().inc``, and ``Counter.get().value``.  The
same shorthand works for getting, setting, or deleting class attributes.

Note, however, that you cannot use this shorthand form when performing
"in-place" operations such as ``+=``, because this would replace the class
with the operation's result.  That is, a statement like ``Counter += 1`` would
delete the ``Counter`` class and replace it with something else!

Therefore, you must always use ``.get()`` before performing an in-place
operation, e.g.::

    service = MyService.get()
    service += 42

This will not change the current instance of the service or delete the class,
but instead will just rebind the variable named ``service`` to point to the
operation's result.  Even this approach is confusing, however, so it's best
to just avoid defining or using in-place operators on your service classes.


Service Replacements
--------------------

You can create alternative implementations for a service, and substitute
them for the original service.  They do *not* have to be subclasses of the
original service for this to work, but they should of course be a suitable
replacement (i.e., implement the same interface), and they *must* use the
``context.replaces()`` class decorator to specify what service they're a
replacement for (even if they are a subclass of that service class).

For example, let's replace our ``Counter`` with an extended version that counts
by twos, and can optionally increment by arbitrary amounts::

    >>> class ExtendedCounter(context.Service):
    ...     context.replaces(Counter)
    ...     value = 0
    ...     def inc(self):
    ...         self.value += 2
    ...     def inc_by(self, increment):
    ...         self.value += increment

    >>> with ExtendedCounter.new():
    ...     print Counter.value
    ...     Counter.inc()
    ...     print Counter.value
    0
    2

    >>> Counter.value
    42

As you can see, our count went back to 0, because inside the ``with`` block,
the current ``Counter`` is actually the ``ExtendedCounter`` instance.  Its
value also went up by 2.  Outside the ``with`` block, the original ``Counter``
instance became current again, and thus ``Counter.value`` now returns the same
value it did before.

Note, by the way, that although ``ExtendedCounter`` has added a new method, it
is not acessible from the ``Counter`` class namespace, even if an
``ExtendedCounter`` is the current counter::

    >>> with ExtendedCounter.new():
    ...     Counter.inc_by(42)
    Traceback (most recent call last):
      ...
    AttributeError: type object 'Counter' has no attribute 'inc_by'

To access such additional methods, you need to either get an instance of the
service using the ``get()`` classmethod, or use the replacement class
directly::

    >>> with ExtendedCounter.new():
    ...     Counter.get().inc_by(42)
    ...     print Counter.value
    ...     ExtendedCounter.inc_by(24)
    ...     print Counter.value
    42
    66

However, the extra method will produce an attribute error at runtime if the
current service instance doesn't actually have that method.  For example, the
code below fails because the default ``Counter`` service is a ``Counter``,
not an ``ExtendedCounter``::

    >>> ExtendedCounter.inc_by
    Traceback (most recent call last):
      ...
    AttributeError: 'Counter' object has no attribute 'inc_by'


Configuring Replacement Services
--------------------------------

Now, you might be wondering at this point how your programs will ever be able
to use an ``ExtendedCounter``, if the default instance is a ``Counter``.  Will
you have to use a huge assortment of ``with`` statements to set up all the
service implementations you want to use?

Fortunately, no, because Contextual provides a configuration system that can be
used to set up service implementations as well as other types of configuration,
using either its built-in ``.ini`` format, or using whatever other mechanism
you might want.  So you can assemble the configuration of your entire program
into a single object, and then use a single ``with`` statement to execute your
program under it.

Another simple way to configure a context's services is to create a new child
context, and register one or more service factories while inside it::

    >>> with context.new():
    ...     Counter <<= ExtendedCounter
    ...     print type(Counter.get())
    <class 'ExtendedCounter'>

The ``<<=`` operator sets the factory to be used for creating a service
instance in the current context.  (It has no effect on any other execution
context.)  The factory can only be set, however, if it hasn't already been used
in the current context.  For example, in the default context, we had already
used ``Counter``, so we can't change the factory to ``ExtendedCounter`` there::

    >>> Counter <<= ExtendedCounter
    Traceback (most recent call last):
      ...
    InputConflict: (<class 'Counter'>, <...'ExtendedCounter'>)

Note, by the way, that the factory does not need to be a class.  It can be any
zero-argument callable, e.g.:

    >>> def a_factory():
    ...     ctr = Counter()
    ...     ctr.value = 99
    ...     return ctr

    >>> with context.new():
    ...     Counter <<= a_factory
    ...     print Counter.value
    99



Configuration and Settings
==========================

XXX Everything from here down is out of date and needs to be rewritten; don't
    read it unless you want to be confused! XXX


Settings
    ``context.setting`` objects provide thread-safe access to stateless
    configuration settings.  Settings consist of a default "input" value, and
    a function for transforming input values to the form needed when using the
    setting.  (For example, the function might take a factory function as
    input, and call it to produce the output, or it might convert strings to
    numeric or date values, etc.)

    Settings can have different input values in different contexts, but once
    the setting's input or output value has been retrieved or computed in a
    given context, it can no longer be changed in that context.  This is a
    major improvement over global variables, as it prevents bugs due to
    changing settings after another part of the program has already used it.

    Of course, this restriction only applies to a single context; each context
    can have a different value for any given setting, and it's easy to create
    a context that inherits its default settings from an existing context (so
    you don't have to copy the settings you want to keep.

Services
    Services are like singletons, but have a separate instance for each thread
    and can easily be replaced as needed for e.g. testing or co-operative
    multitasking.  Services are created by subclassing ``context.Service``, and
    the resulting class's ``get()`` classmethod will return the current
    instance.  Alternative implementations of a service can be defined and used
    to replace the default implementation in a given context.

    Services are also similar to settings, in that they take an input value
    (which must be 

    (But as with
    settings, the choice of replacement cannot be changed in a context where
    the

    , but as with settings, the choice of
    implementation cannot be changed within

    Services also work like settings, in that you cannot  whose input value is a class or
    factory for creating instances of the service, and its

Transaction Resources
    The ``context.Action`` service manages access to objects that may need to
    be recycled, saved, or reverted at the completion of an action such as
    a transaction, command, web request, or other atomic operation.  Requesting
    a resource (such as a database connection) causes it to be registered with
    the current ``Action``, so that it will be notified of the success or
    failure of the action.  This then allows the
    resource to commit or rollback as appropriate, return connections to
    connection pools or close them, etc.

    The boundaries of an action are indicated by the scope of a ``with:``
    statement (or substitute).  E.g.::

        from peak.context import Action

        @Action.resource
        def db_conn(value="default connection string"):
            conn = db_open(value)
            return conn

        with context.Action.new():
            # any resources used here will be notified of the
            # success or failure of this block of code

            # for example, the opened db connection will get __enter__
            # called here:
            c = db_conn()
            # and then its __exit__ will be called when the block is finished

In addition, ``Contextual`` offers some related convenience features:

``State`` objects
    State objects can be used to explicitly manipulate the current context,
    e.g. to swap out the current state of all settings, resources, and
    services, thereby creating either a new blank slate (e.g.
    for tests) or swapping between multiple independent tasks in the same
    thread (e.g. to do co-operative multitasking).

Registries
    Normally, configuration settings and action resources are defined
    statically in code.  But sometimes you want to create a dynamic namespace
    where any name could potentially identify a valid setting or resource,
    possibly with some kind of rules to compute default values when they
    haven't been explicitly set.  The ``registry`` and ``resource_registry``
    decorators let you create such namespaces for settings and resources.

Configuration Files
    ``Contextual`` offers an extensible ``.ini``-style configuration file
    format that can be used to configure arbitrary settings and resources
    across any number of modules, thereby giving you global configuration
    of an entire application from a single configuration system.

    The loaded configuration is lazy, which means that it doesn't force
    importing of any of the modules being configured, and doesn't compute any
    values until they're actually needed.  Thus, a configuration file can
    include optional settings for modules that may not even be installed!

    Although the ``.ini`` syntax is a fixed property of the configuration
    loader, the semantics can be extended to create custom section handlers
    that interpret their contents differently -- and this can even be done
    from within the ``.ini`` files themselves, because the handlers are looked
    up using the same configuration services that everything else uses.

"with"-statement Emulation and Tools
    Finally, ``Contextual`` offers a variety of decorators, base classes, and
    utility functions that are useful for either emulating the "with" statement
    in older versions of Python, and/or working with it in Python 2.5+.






Actions and Resources
=====================


Namespaces and Configuration Files
==================================


Dynamic Parameters
==================


The Application Object
======================


Co-operative Multi-tasking
--------------------------


-------------
API Reference
-------------

Service
replaces()

@value
@resource
@expression

Scope

using, only, State

InputConflict
DynamicRuleError

Action

@namespace

lookup


Support for Python 2.3/2.4
==========================

The "with" statement is only available in Python 2.5 and up, but Contextual is
intended to be usable with Python 2.3 and up.  So, Contextual include a few
functions that let you emulate most of the "with" statement's functionality
in Python 2.3 and 2.4.  You should use them in place of the Python 2.5 features
if you want your code to work on Python 2.3 or 2.4.

Here's a simple translation table::

    Python 2.5+               Python 2.4 (for 2.3, put decorators after funcs)
    ------------              ------------------------------------------------
    with x as y:              @context.call_with(x)
        print y               def do_it(y):
                                  print y

    with x as y:              z = context.with_(x,f)
        z = f(y)

    @contextmanager           @context.manager
    def transacted():         def transacted():
        begin_txn()               begin_txn()
        try:                      try:
            yield                     yield None
        except:                       context.reraise()
            abort_txn()           except:
            raise                     abort_txn()
        else:                         raise
            commit_txn()          else:
                                      commit_txn()

The biggest difference between these constructs (apart from performance) is
that the "with" statement lets you rebind variables in the surrounding scope,
because the code in the block is still in the same scope.  Since our emulation
for Python 2.4 uses functions, they can't rebind variables from the surrounding
scope, and thus must make do with side effects and/or return values.

Notice also that we allow definition of context managers using generators, much
like in Python 2.5, but you must *always* call ``context.reraise()`` following
your ``yield None``, so that any error raised by the "with" block will be
propagated.  (Unless, of course, you want to suppress it.)