The new algorithm lands - version bump to 0.6a1dev. Too many changes to list!
================================================= Software Transactional Memory (STM) And Observers ================================================= The Trellis is built on a simple Python STM (Software Transactional Memory) and "Observer Pattern" implementation. This document specifies how that implementation works, and tests it. You should read this document if you plan to implement your own Trellis cell types or other custom data structures, or if you just want to know how things work "under the hood". STM History =========== An STM's job is to manage "atomic" changes to objects: i.e., multiple changes that must happen as a unit, or else be rolled back. To do this, the STM must have a **history**: a record of the actions taken within a given "atomic" change set. The PEAK implementation of an STM history is found in the ``peak.events.stm`` module:: >>> from peak.events.stm import STMHistory >>> hist = STMHistory() A history object's ``atomically()`` method invokes a function as an atomic operation, and its ``active`` attribute indicates whether it is currently performing an atomic operation:: >>> def is_active(): ... print "Active?", hist.active >>> is_active() Active? False >>> hist.atomically(is_active) Active? True >>> is_active() Active? False Nested calls to ``atomically()`` simply execute the function, without affecting the history:: >>> def nested_operation(): ... hist.atomically(is_active) ... is_active() >>> hist.atomically(nested_operation) Active? True Active? True Commit/Abort Notices -------------------- In order to get notification of commits or aborts, you can register Python "context managers" with the history, using the ``manage()`` method. The ``manage()`` method takes a context manager and calls its ``__enter__()`` method immediately, then calls its ``__exit__()`` method when the current atomic operation is completed. This allows things like locks or other resources to be acquired as the operation progresses, and then get automatically released when the operation is completed:: >>> class DemoManager(object): ... def __init__(self, num): ... self.num = num ... def __enter__(self): ... print "Manager", self.num, "entering" ... def __exit__(self, typ, val, tb): ... print "Manager", self.num, "exiting", typ, val, tb >>> hist.manage(DemoManager(1)) Traceback (most recent call last): ... AssertionError: Can't manage without active history >>> hist.atomically(hist.manage, DemoManager(2)) Manager 2 entering Manager 2 exiting None None None The same context manager can be passed to ``manage`` repeatedly, but its enter/exit methods will only be called once:: >>> def multi_manage(): ... mgr = DemoManager(3) ... hist.manage(mgr) ... hist.manage(mgr) >>> hist.atomically(multi_manage) Manager 3 entering Manager 3 exiting None None None And if multiple context managers are registered, their ``__exit__`` methods are called in the opposite order from their ``__enter__`` methods:: >>> def multi_manage(): ... hist.manage(DemoManager(4)) ... hist.manage(DemoManager(5)) >>> hist.atomically(multi_manage) Manager 4 entering Manager 5 entering Manager 5 exiting None None None Manager 4 exiting None None None The ``__exit__()`` method is normally called with three ``None`` values, unless an exception occurs during the operation. In that case, the ``sys.exc_info()`` of the exception is passed in to the manager(s), before the exception is re-raised:: >>> def do_error(): ... hist.manage(DemoManager(6)) ... raise TypeError("Testing!") >>> try: ... hist.atomically(do_error) ... except TypeError: ... print "caught exception" Manager 6 entering Manager 6 exiting <type '...TypeError'> Testing! <traceback object...> caught exception The ``__exit__()`` method should not raise an error. Also note that, unlike normal Python context managers, the return value of ``__exit__()`` is ignored. (In other words, STM context managers cannot cause an exception to be ignored.) If an ``__exit__()`` method *does* raise an error, however, subsequent context managers will be passed the type, value, and traceback of the failing manager, and the exception will be reraised:: >>> class ErrorManager(DemoManager): ... def __exit__(self, typ, val, tb): ... super(ErrorManager, self).__exit__(typ, val, tb) ... raise RuntimeError("Haha!") >>> def manage_with_error(): ... hist.manage(DemoManager(7)) ... hist.manage(ErrorManager("error")) ... hist.manage(DemoManager(8)) >>> try: ... hist.atomically(manage_with_error) ... except RuntimeError: ... print "caught exception" Manager 7 entering Manager error entering Manager 8 entering Manager 8 exiting None None None Manager error exiting None None None Manager 7 exiting <type '...RuntimeError'> Haha! <traceback object...> caught exception In other words, all context managers are guaranteed to have their ``__exit__`` methods called, even if one fails. The exception that comes out of the ``atomically()`` call will be the most recently-raised exception. Last, but not least, history objects have a ``in_cleanup`` attribute that indicates whether they are currently in the process of comitting or aborting an operation. This can be useful if context managers might call code that needs to behave differently during a commit/abort than during an atomic operation:: >>> hist.in_cleanup False >>> class CleanupManager: ... def __enter__(self): ... print "on entry:", hist.in_cleanup ... def __exit__(self, typ, val, tb): ... print "on exit:", hist.in_cleanup >>> hist.atomically(hist.manage, CleanupManager()) on entry: False on exit: True Undo, Rollback and Save Points ------------------------------ While you can use context managers to implement some forms of commit/rollback, it's easier for most things to use a history object's "undo" log. The log records "undo actions": functions (and optional positional arguments) that can be used to undo whatever operations have been done so far. For example:: >>> def undoing(msg): ... print "undoing", msg >>> def with_undo(): ... hist.on_undo(undoing, "op 1") ... hist.on_undo(undoing, "op 2") >>> hist.atomically(with_undo) # success, nothing gets undone Nothing happened here, because the operation completed successfully. But if an error occurs, the undo functions are called in reverse order:: >>> def with_undo(): ... hist.on_undo(undoing, "op 1") ... hist.on_undo(undoing, "op 2") ... raise TypeError("foo") >>> try: ... hist.atomically(with_undo) ... except TypeError: ... print "caught exception" undoing op 2 undoing op 1 caught exception But this does NOT happen if the error occurs in a manager's ``__exit__()`` method:: >>> def with_undo(): ... hist.manage(ErrorManager("error")) ... hist.on_undo(undoing, "op 1") ... hist.on_undo(undoing, "op 2") >>> try: ... hist.atomically(with_undo) ... except RuntimeError: ... print "caught exception" Manager error entering Manager error exiting None None None caught exception (That's because managers may be used to control locking or other context setups that need to still be in place for the undo's to be safe.) Note, by the way, that undo functions must NEVER raise errors, under any circumstances. If they do, any undo functions that have *not* been called yet, will never *be* called. In addition to the automatic rollback on error, you can also record savepoints within an atomic operation, and then rollback to that savepoint at any time later:: >>> def with_savepoint(): ... hist.on_undo(undoing, "op 1") ... sp = hist.savepoint() ... hist.on_undo(undoing, "op 2") ... hist.on_undo(undoing, "op 3") ... hist.rollback_to(sp) >>> hist.atomically(with_savepoint) undoing op 3 undoing op 2 Commit Callbacks ---------------- If you want to defer an action until the overal operation is ready to commit, you can use the ``on_commit(func, *args)`` method:: >>> def do_something(msg): ... print "committing", msg >>> def do_with_commit(): ... hist.on_commit(do_something, "hello!") ... print "registered commit operation" >>> hist.atomically(do_with_commit) registered commit operation committing hello! Commit actions are called in the order they are registered (before any managers) and their registrations are subject to undo:: >>> def do_with_commit_and_undo(): ... hist.manage(DemoManager('test')) ... hist.on_commit(do_something, 1) ... sp = hist.savepoint() ... hist.on_commit(do_something, 2) ... hist.rollback_to(sp) ... hist.on_commit(do_something, 3) >>> hist.atomically(do_with_commit_and_undo) Manager test entering committing 1 committing 3 Manager test exiting None None None Commit actions are also allowed to record undo actions, in case there is an error later in the commit process:: >>> def do_with_commit_and_undo(): ... def f1(): ... print "f1 running" ... hist.on_undo(f3) ... def f2(): ... print "f2 running" ... raise AssertionError ... def f3(): ... print "f3 running" ... hist.on_commit(f1) ... hist.on_commit(f2) >>> try: ... hist.atomically(do_with_commit_and_undo) ... except AssertionError: ... print "caught exception" f1 running f2 running f3 running caught exception And of course, commit actions are not run if an error occurs before they have a chance to run: >>> def do_no_commit(): ... hist.on_commit(do_something, "should not happen") ... raise AssertionError >>> try: ... hist.atomically(do_no_commit) ... except AssertionError: ... print "caught exception" caught exception Logged Setattr -------------- As a convenience, you can use the ``setattr()`` method of history objects to automatically log an undo function to restore the old value of the attribute being set:: >>> class SomeObject: ... pass >>> s1 = SomeObject() >>> s1.foo = 'bar' >>> def setattr_normally(): ... hist.change_attr(s1, 'foo', "baz") >>> hist.atomically(setattr_normally) >>> s1.foo 'baz' >>> def setattr_rollback(): ... hist.change_attr(s1, 'foo', "spam") ... raise TypeError >>> hist.atomically(setattr_rollback) Traceback (most recent call last): ... TypeError >>> s1.foo # result is unchanged after rollback 'baz' As you can see, this saves you the work of recording the undo operation manually. The Observer Framework ====================== Above and beyond the bare STM system, the Trellis also needs a strong "observer" system that manages subjects, listeners, and the links between them:: >>> from peak.events import stm Subjects, Listeners, and Links ------------------------------ The ``AbstractSubject`` and ``AbstractListener`` base classes are used to implement the observer pattern. They must be subclassed to be used. If you will be using a lot of instances, you can conserve memory by using ``__slots__`` in your subclass to store the needed attributes, as shown:: >>> class Listener(stm.AbstractListener): ... __slots__ = '__weakref__', 'layer', 'next_subject' >>> class Subject(stm.AbstractSubject): ... __slots__ = 'layer', 'next_listener' Listeners and Subjects can be many-to-many related. That is, each listener may listen to zero or more subjects, and each subject can have zero or more listeners. These relationships are created using ``Link`` objects:: >>> l1 = Listener() >>> l2 = Listener() >>> s1 = Subject() >>> s2 = Subject() >>> link11 = stm.Link(s1, l1) >>> link21 = stm.Link(s2, l1) >>> link12 = stm.Link(s1, l2) >>> link22 = stm.Link(s2, l2) >>> list(l1.iter_subjects())==list(l2.iter_subjects())==[s2, s1] True >>> list(s1.iter_listeners())==list(s2.iter_listeners())==[l2, l1] True The relationship between a subject and its listeners is "weak"; that is, if a listener goes out of existence, all of the relevant links are unlinked from the relevant subjects' listener chains:: >>> del l1 >>> list(s1.iter_listeners())==list(s2.iter_listeners())==[l2] True You can also explicitly break a link by calling its ``.unlink()`` method:: >>> link12.unlink() >>> list(s1.iter_listeners()) [] >>> list(s2.iter_listeners())==[l2] True Normally, however, you will not create or manage links explicitly, as they are automatically managed by the controller.
cvs-admin@eby-sarna.com Powered by ViewCVS 1.0-dev |
ViewCVS and CVS Help |