[Subversion] / BytecodeAssembler / README.txt  

Diff of /BytecodeAssembler/README.txt

Parent Directory | Revision Log

version 2209, Wed Jul 5 05:46:10 2006 UTC version 2732, Sat Apr 4 00:47:53 2015 UTC
Line 9 
Line 9 
 bytecode instead of on these mechanical issues.  bytecode instead of on these mechanical issues.
   
 In addition to a low-level opcode-oriented API for directly generating specific  In addition to a low-level opcode-oriented API for directly generating specific
 bytecodes, this module also offers an extensible mini-AST framework for  Python bytecodes, this module also offers an extensible mini-AST framework for
 generating code from high-level specifications.  This framework does most of  generating code from high-level specifications.  This framework does most of
 the work needed to transform tree-like structures into linear bytecode  the work needed to transform tree-like structures into linear bytecode
 instructions, and includes the ability to do compile-time constant folding.  instructions, and includes the ability to do compile-time constant folding.
   
   Please see the `BytecodeAssembler reference manual`_ for more details.
   
   .. _BytecodeAssembler reference manual: http://peak.telecommunity.com/DevCenter/BytecodeAssembler#toc
   
   
   Changes since version 0.5.2:
   
   * Symbolic disassembly with full emulation of backward-compatible
     ``JUMP_IF_TRUE`` and ``JUMP_IF_FALSE`` opcodes on Python 2.7 -- tests now
     run clean on Python 2.7.
   
   * Support for backward emulation of Python 2.7's ``JUMP_IF_TRUE_OR_POP`` and
     ``JUMP_IF_FALSE_OR_POP`` instructions on earlier Python versions; these
     emulations are also used in BytecodeAssembler's internal code generation,
     for maximum performance on 2.7+ (with no change to performance on older
     versions).
   
   Changes since version 0.5.1:
   
   * Initial support for Python 2.7's new opcodes and semantics changes, mostly
     by emulating older versions' behavior with macros.  (0.5.2 is really just
     a quick-fix release to allow packages using BytecodeAssembler to run on 2.7
     without having to change any of their code generation; future releases will
     provide proper support for the new and changed opcodes, as well as a test
     suite that doesn't show spurious differences in the disassembly listings
     under Python 2.7.)
   
   Changes since version 0.5:
   
   * Fix incorrect stack size calculation for ``MAKE_CLOSURE`` on Python 2.5+
   
   Changes since version 0.3:
   
   * New node types:
   
     * ``For(iterable, assign, body)`` -- define a "for" loop over `iterable`
   
     * ``UnpackSequence(nodes)`` -- unpacks a sequence that's ``len(nodes)`` long,
       and then generates the given nodes.
   
     * ``LocalAssign(name)`` -- issues a ``STORE_FAST``, ``STORE_DEREF`` or
       ``STORE_LOCAL`` as appropriate for the given name.
   
     * ``Function(body, name='<lambda>', args=(), var=None, kw=None, defaults=())``
       -- creates a nested function from `body` and puts it on the stack.
   
     * ``If(cond, then_, else_=Pass)`` -- "if" statement analogue
   
     * ``ListComp(body)`` and ``LCAppend(value)`` -- implement list comprehensions
   
     * ``YieldStmt(value)`` -- generates a ``YIELD_VALUE`` (plus a ``POP_TOP`` in
       Python 2.5+)
   
   * ``Code`` objects are now iterable, yielding ``(offset, op, arg)`` triples,
     where `op` is numeric and `arg` is either numeric or ``None``.
   
   * ``Code`` objects' ``.code()`` method can now take a "parent" ``Code`` object,
     to link the child code's free variables to cell variables in the parent.
   
   * Added ``Code.from_spec()`` classmethod, that initializes a code object from a
     name and argument spec.
   
   * ``Code`` objects now have a ``.nested(name, args, var, kw)`` method, that
     creates a child code object with the same ``co_filename`` and the supplied
     name/arg spec.
   
   * Fixed incorrect stack tracking for the ``FOR_ITER`` and ``YIELD_VALUE``
     opcodes
   
   * Ensure that ``CO_GENERATOR`` flag is set if ``YIELD_VALUE`` opcode is used
   
   * Change tests so that Python 2.3's broken line number handling in ``dis.dis``
     and constant-folding optimizer don't generate spurious failures in this
     package's test suite.
   
   
   Changes since version 0.2:
   
   * Added ``Suite``, ``TryExcept``, and ``TryFinally`` node types
   
   * Added a ``Getattr`` node type that does static or dynamic attribute access
     and constant folding
   
   * Fixed ``code.from_function()`` not copying the ``co_filename`` attribute when
     ``copy_lineno`` was specified.
   
   * The ``repr()`` of AST nodes doesn't include a trailing comma for 1-argument
     node types any more.
   
   * Added a ``Pass`` symbol that generates no code, a ``Compare()`` node type
     that does n-way comparisons, and ``And()`` and ``Or()`` node types for doing
     logical operations.
   
   * The ``COMPARE_OP()`` method now accepts operator strings like ``"<="``,
     ``"not in"``, ``"exception match"``, and so on, as well as numeric opcodes.
     See the standard library's ``opcode`` module for a complete list of the
     strings accepted (in the ``cmp_op`` tuple).  ``"<>"`` is also accepted as an
     alias for ``"!="``.
   
   * Added code to verify that forward jump offsets don't exceed a 64KB span, and
     support absolute backward jumps to locations >64KB.
   
 Changes since version 0.1:  Changes since version 0.1:
   
 * Constant handling has been fixed so that it doesn't confuse equal values of  * Constant handling has been fixed so that it doesn't confuse equal values of
   differing types (e.g. ``1.0`` and ``True``), or equal unhashable objects    differing types (e.g. ``1.0`` and ``True``), or equal unhashable objects
   (e.g. two empty lists).    (e.g. two empty lists).
   
 * Removed ``nil`, ``ast_curry()`` and ``folding_curry()``, replacing them with  * Removed ``nil``, ``ast_curry()`` and ``folding_curry()``, replacing them with
   the ``nodetype()`` decorator and ``fold_args()``; please see the docs for    the ``nodetype()`` decorator and ``fold_args()``; please see the docs for
   more details.    more details.
   
Line 50 
Line 152 
 * Various bug fixes  * Various bug fixes
   
 There are a few features that aren't tested yet, and not all opcodes may be  There are a few features that aren't tested yet, and not all opcodes may be
 fully supported.  Notably, the following features are still NOT reliably  fully supported.  Also note the following limitations:
 supported yet:  
   
 * Wide jump addressing (for generated bytecode>64K in size)  * Jumps to as-yet-undefined labels cannot span a distance greater than 65,535
     bytes.
   
 * The ``dis()`` module in Python 2.3 has a bug that makes it show incorrect  * The ``dis()`` function in Python 2.3 has a bug that makes it show incorrect
   line numbers when the difference between two adjacent line numbers is    line numbers when the difference between two adjacent line numbers is
   greater than 255.  This causes two shallow failures in the current test    greater than 255.  (To work around this, the test_suite uses a later version
   suite when it's run under Python 2.3.    of ``dis()``, but do note that it may affect your own tests if you use
     ``dis()`` with Python 2.3 and use widely separated line numbers.)
   
 If you find any other issues, please let me know.  If you find any other issues, please let me know.
   
Line 68 
Line 171 
 Questions and discussion regarding this software should be directed to the  Questions and discussion regarding this software should be directed to the
 `PEAK Mailing List <http://www.eby-sarna.com/mailman/listinfo/peak>`_.  `PEAK Mailing List <http://www.eby-sarna.com/mailman/listinfo/peak>`_.
   
   .. _toc:
 .. contents:: **Table of Contents**  .. contents:: **Table of Contents**
   
   
Line 106 
Line 210 
 that maps each ``set_lineno()`` to the corresponding position in the bytecode.  that maps each ``set_lineno()`` to the corresponding position in the bytecode.
   
 And of course, the resulting code objects can be run with ``eval()`` or  And of course, the resulting code objects can be run with ``eval()`` or
 ``exec``, or used with ``new.function`` to create a function::  ``exec``, or used with ``new.function``/``types.FunctionType`` to create a
   function::
   
     >>> eval(c.code())      >>> eval(c.code())
     42      42
   
     >>> exec c.code()   # exec discards the return value, so no output here      >>> exec(c.code())   # exec discards the return value, so no output here
   
       >>> try:
       ...     from new import function
       ... except ImportError:  # Python 3 workarounds
       ...     from types import FunctionType as function
       ...     long = int
       ...     unicode = str
   
     >>> import new      >>> f = function(c.code(), globals())
     >>> f = new.function(c.code(), globals())  
     >>> f()      >>> f()
     42      42
   
   Finally, code objects are also iterable, yielding ``(offset, opcode, arg)``
   tuples, where `arg` is ``None`` for opcodes with no arguments, and an integer
   otherwise::
   
       >>> import peak.util.assembler as op
       >>> list(c) == [
       ...     (0, op.LOAD_CONST, 1),
       ...     (3, op.RETURN_VALUE, None)
       ... ]
       True
   
   This can be useful for testing or otherwise inspecting code you've generated.
   
   
   Symbolic Disassembler
   =====================
   
   Python's built-in disassembler can be verbose and hard to read when inspecting
   complex generated code -- usually you don't care about bytecode offsets or
   line numbers as much as you care about labels, for example.
   
   So, BytecodeAssembler provides its own, simplified disassembler, which we'll
   be using for more complex listings in this manual::
   
       >>> from peak.util.assembler import dump
   
   Some sample output, that also showcases some of BytecodeAssembler's
   `High-Level Code Generation`_ features::
   
       >>> c = Code()
       >>> from peak.util.assembler import Compare, Local
       >>> c.return_(Compare(Local('a'), [('<', Local('b')), ('<', Local('c'))]))
       >>> dump(c.code())
                       LOAD_FAST                0 (a)
                       LOAD_FAST                1 (b)
                       DUP_TOP
                       ROT_THREE
                       COMPARE_OP               0 (<)
                       JUMP_IF_FALSE           L1
                       POP_TOP
                       LOAD_FAST                2 (c)
                       COMPARE_OP               0 (<)
                       JUMP_FORWARD            L2
               L1:     ROT_TWO
                       POP_TOP
               L2:     RETURN_VALUE
   
   As you can see, the line numbers and bytecode offsets have been dropped,
   making it esier to see where the jumps go.  (This also makes doctests more
   robust against Python version changes, as ``dump()`` has some extra code to
   make conditional jumps appear consistent across the major changes that were
   made to conditional jump instructions between Python 2.6 and 2.7.)
   
   
 Opcodes and Arguments  Opcodes and Arguments
 =====================  =====================
Line 182 
Line 346 
     >>> c.RETURN_VALUE()      >>> c.RETURN_VALUE()
   
     >>> eval(c.code())          # computes type(27)      >>> eval(c.code())          # computes type(27)
     <type 'int'>      <... 'int'>
   
     >>> c = Code()      >>> c = Code()
     >>> c.LOAD_CONST(dict)      >>> c.LOAD_CONST(dict)
Line 211 
Line 375 
     >>> c.POP_TOP()      >>> c.POP_TOP()
     >>> c.JUMP_ABSOLUTE(where)   # now jump back to it      >>> c.JUMP_ABSOLUTE(where)   # now jump back to it
   
     >>> dis(c.code())      >>> dump(c.code())
       0           0 LOAD_CONST               1 (42)                      LOAD_CONST               1 (42)
             >>    3 DUP_TOP              L1:     DUP_TOP
                   4 POP_TOP                      POP_TOP
                   5 JUMP_ABSOLUTE            3                      JUMP_ABSOLUTE            L1
   
 But if you are jumping *forward*, you will need to call the jump or setup  But if you are jumping *forward*, you will need to call the jump or setup
 method without any arguments.  The return value will be a "forward reference"  method without any arguments.  The return value will be a "forward reference"
Line 233 
Line 397 
     >>> c.LOAD_CONST(23)      >>> c.LOAD_CONST(23)
     >>> c.RETURN_VALUE()      >>> c.RETURN_VALUE()
   
     >>> dis(c.code())      >>> dump(c.code())
       0           0 LOAD_CONST               1 (99)                      LOAD_CONST               1 (99)
                   3 JUMP_IF_TRUE             4 (to 10)                      JUMP_IF_TRUE             L1
                   6 LOAD_CONST               2 (42)                      LOAD_CONST               2 (42)
                   9 POP_TOP                      POP_TOP
             >>   10 LOAD_CONST               3 (23)              L1:     LOAD_CONST               3 (23)
                  13 RETURN_VALUE                      RETURN_VALUE
   
     >>> eval(c.code())      >>> eval(c.code())
     23      23
Line 263 
Line 427 
     >>> c = Code()      >>> c = Code()
     >>> c.co_cellvars = ('a','b')      >>> c.co_cellvars = ('a','b')
   
       >>> import sys
     >>> c.LOAD_CLOSURE('a')      >>> c.LOAD_CLOSURE('a')
     >>> c.LOAD_CLOSURE('b')      >>> c.LOAD_CLOSURE('b')
       >>> if sys.version>='2.5':
       ...     c.BUILD_TUPLE(2) # In Python 2.5+, free vars must be in a tuple
     >>> c.LOAD_CONST(None)  # in real code, this'd be a Python code constant      >>> c.LOAD_CONST(None)  # in real code, this'd be a Python code constant
     >>> c.MAKE_CLOSURE(0,2) # no defaults, 2 free vars in the new function      >>> c.MAKE_CLOSURE(0,2) # no defaults, 2 free vars in the new function
   
       >>> c.stack_size         # This will be 1, no matter what Python version
       1
   
   The ``COMPARE_OP`` method takes an argument which can be a valid comparison
   integer constant, or a string containing a Python operator, e.g.::
   
       >>> c = Code()
       >>> c.LOAD_CONST(1)
       >>> c.LOAD_CONST(2)
       >>> c.COMPARE_OP('not in')
       >>> dis(c.code())
         0           0 LOAD_CONST               1 (1)
                     3 LOAD_CONST               2 (2)
                     6 COMPARE_OP               7 (not in)
   
   The full list of valid operator strings can be found in the standard library's
   ``opcode`` module.  ``"<>"`` is also accepted as an alias for ``"!="``::
   
       >>> c.LOAD_CONST(3)
       >>> c.COMPARE_OP('<>')
       >>> dis(c.code())
         0           0 LOAD_CONST               1 (1)
                     3 LOAD_CONST               2 (2)
                     6 COMPARE_OP               7 (not in)
                     9 LOAD_CONST               3 (3)
                    12 COMPARE_OP               3 (!=)
   
   
 High-Level Code Generation  High-Level Code Generation
 ==========================  ==========================
Line 288 
Line 482 
 ``None``, or Python code object, it is treated as though it was passed to  ``None``, or Python code object, it is treated as though it was passed to
 the ``LOAD_CONST`` method directly::  the ``LOAD_CONST`` method directly::
   
   
   
     >>> c = Code()      >>> c = Code()
     >>> c(1, 2L, 3.0, 4j+5, "6", u"7", False, None, c.code())      >>> c(1, long(2), 3.0, 4j+5, "6", unicode("7"), False, None, c.code())
     >>> dis(c.code())      >>> dis(c.code())
       0           0 LOAD_CONST               1 (1)        0           0 LOAD_CONST               1 (1)
                   3 LOAD_CONST               2 (2L)                    3 LOAD_CONST               2 (2...)
                   6 LOAD_CONST               3 (3.0)                    6 LOAD_CONST               3 (3.0)
                   9 LOAD_CONST               4 ((5+4j))                    9 LOAD_CONST               4 ((5+4j))
                  12 LOAD_CONST               5 ('6')                   12 LOAD_CONST               5 ('6')
                  15 LOAD_CONST               6 (u'7')                   15 LOAD_CONST               6 (...'7')
                  18 LOAD_CONST               7 (False)                   18 LOAD_CONST               7 (False)
                  21 LOAD_CONST               0 (None)                   21 LOAD_CONST               0 (None)
                  24 LOAD_CONST               8 (<code object <lambda> at ...>)                   24 LOAD_CONST               8 (<code object <lambda>...>)
   
 Note that although some values of different types may compare equal to each  Note that although some values of different types may compare equal to each
 other, ``Code`` objects will not substitute a value of a different type than  other, ``Code`` objects will not substitute a value of a different type than
 the one you requested::  the one you requested::
   
     >>> c = Code()      >>> c = Code()
     >>> c(1, True, 1.0, 1L)     # equal, but different types      >>> c(1, True, 1.0)     # equal, but different types
     >>> dis(c.code())      >>> dis(c.code())
       0           0 LOAD_CONST               1 (1)        0           0 LOAD_CONST               1 (1)
                   3 LOAD_CONST               2 (True)                    3 LOAD_CONST               2 (True)
                   6 LOAD_CONST               3 (1.0)                    6 LOAD_CONST               3 (1.0)
                   9 LOAD_CONST               4 (1L)  
   
   
 Simple Containers  Simple Containers
 -----------------  -----------------
Line 386 
Line 580 
     False      False
   
   
   ``Suite`` and ``Pass``
   ----------------------
   
   On occasion, it's helpful to be able to group a sequence of opcodes,
   expressions, or statements together, to be passed as an argument to other node
   types.  The ``Suite`` node type accomplishes this::
   
       >>> from peak.util.assembler import Suite, Pass
   
       >>> c = Code()
       >>> c.return_(Suite([Const(42), Code.DUP_TOP, Code.POP_TOP]))
       >>> dis(c.code())
         0           0 LOAD_CONST               1 (42)
                     3 DUP_TOP
                     4 POP_TOP
                     5 RETURN_VALUE
   
   And ``Pass`` is a shortcut for an empty ``Suite``, that generates nothing::
   
       >>> Suite([])
       Pass
   
       >>> c = Code()
       >>> c(Pass)
       >>> c.return_(None)
       >>> dis(c.code())
         0           0 LOAD_CONST               0 (None)
                     3 RETURN_VALUE
   
   
 Local and Global Names  Local and Global Names
 ----------------------  ----------------------
   
Line 400 
Line 624 
       0           0 LOAD_FAST                0 (x)        0           0 LOAD_FAST                0 (x)
                   3 LOAD_GLOBAL              0 (y)                    3 LOAD_GLOBAL              0 (y)
   
   
 As with simple constants and ``Const`` wrappers, these objects can be used to  As with simple constants and ``Const`` wrappers, these objects can be used to
 construct more complex expressions, like ``{a:(b,c)}``::  construct more complex expressions, like ``{a:(b,c)}``::
   
Line 416 
Line 639 
                  16 ROT_THREE                   16 ROT_THREE
                  17 STORE_SUBSCR                   17 STORE_SUBSCR
   
   The ``LocalAssign`` node type takes a name, and stores a value in a local
   variable::
   
       >>> from peak.util.assembler import LocalAssign
       >>> c = Code()
       >>> c(42, LocalAssign('x'))
       >>> dis(c.code())
         0           0 LOAD_CONST               1 (42)
                     3 STORE_FAST               0 (x)
   
 If the code object is not using "fast locals" (i.e. ``CO_OPTIMIZED`` isn't  If the code object is not using "fast locals" (i.e. ``CO_OPTIMIZED`` isn't
 set), local variables will be dereferenced using ``LOAD_NAME`` instead of  set), local variables will be referenced using ``LOAD_NAME`` and ``STORE_NAME``
 ``LOAD_FAST``, and if the referenced local name is a "cell" or "free"  instead of ``LOAD_FAST`` and ``STORE_FAST``, and if the referenced local name
 variable, ``LOAD_DEREF`` is used instead::  is a "cell" or "free" variable, ``LOAD_DEREF`` and ``STORE_DEREF`` are used
   instead::
   
     >>> from peak.util.assembler import CO_OPTIMIZED      >>> from peak.util.assembler import CO_OPTIMIZED
     >>> c = Code()      >>> c = Code()
Line 427 
Line 661 
     >>> c.co_cellvars = ('y',)      >>> c.co_cellvars = ('y',)
     >>> c.co_freevars = ('z',)      >>> c.co_freevars = ('z',)
     >>> c( Local('x'), Local('y'), Local('z') )      >>> c( Local('x'), Local('y'), Local('z') )
       >>> c( LocalAssign('x'), LocalAssign('y'), LocalAssign('z') )
     >>> dis(c.code())      >>> dis(c.code())
       0           0 LOAD_NAME                0 (x)        0           0 LOAD_NAME                0 (x)
                   3 LOAD_DEREF               0 (y)                    3 LOAD_DEREF               0 (y)
                   6 LOAD_DEREF               1 (z)                    6 LOAD_DEREF               1 (z)
                     9 STORE_NAME               0 (x)
                    12 STORE_DEREF              0 (y)
                    15 STORE_DEREF              1 (z)
   
   
   Obtaining Attributes
   --------------------
   
   The ``Getattr`` node type takes an expression and an attribute name.  The
   attribute name can be a constant string, in which case a ``LOAD_ATTR`` opcode
   is used, and constant folding is done if possible::
   
       >>> from peak.util.assembler import Getattr
   
       >>> c = Code()
       >>> c(Getattr(Local('x'), '__class__'))
       >>> dis(c.code())
         0           0 LOAD_FAST                0 (x)
                     3 LOAD_ATTR                0 (__class__)
   
   
       >>> Getattr(Const(object), '__class__') # const expression, const result
       Const(<... 'type'>)
   
   Or the attribute name can be an expression, in which case a ``getattr()`` call
   is compiled instead::
   
       >>> c = Code()
       >>> c(Getattr(Local('x'), Local('y')))
       >>> dis(c.code())
         0           0 LOAD_CONST               1 (<built-in function getattr>)
                     3 LOAD_FAST                0 (x)
                     6 LOAD_FAST                1 (y)
                     9 CALL_FUNCTION            2
   
   
 Calling Functions and Methods  Calling Functions and Methods
Line 512 
Line 781 
                   3 RETURN_VALUE                    3 RETURN_VALUE
   
   
   ``If`` Conditions
   -----------------
   
   The ``If()`` node type generates conditional code, roughly equivalent to a
   Python if/else statement::
   
       >>> from peak.util.assembler import If
       >>> c = Code()
       >>> c( If(Local('a'), Return(42), Return(55)) )
       >>> dump(c.code())
                       LOAD_FAST                0 (a)
                       JUMP_IF_FALSE            L1
                       POP_TOP
                       LOAD_CONST               1 (42)
                       RETURN_VALUE
               L1:     POP_TOP
                       LOAD_CONST               2 (55)
                       RETURN_VALUE
   
   However, it can also be used like a Python 2.5+ conditional expression
   (regardless of the targeted Python version)::
   
       >>> c = Code()
       >>> c( Return(If(Local('a'), 42, 55)) )
       >>> dump(c.code())
                       LOAD_FAST                0 (a)
                       JUMP_IF_FALSE            L1
                       POP_TOP
                       LOAD_CONST               1 (42)
                       JUMP_FORWARD             L2
               L1:     POP_TOP
                       LOAD_CONST               2 (55)
               L2:     RETURN_VALUE
   
   
   Note that ``If()`` does *not* do constant-folding on its condition; even if the
   condition is a constant, it will be tested at runtime.  This avoids issues with
   using mutable constants, e.g.::
   
       >>> c = Code()
       >>> c(If(Const([]), 42, 55))
       >>> dump(c.code())
                       LOAD_CONST               1 ([])
                       JUMP_IF_FALSE            L1
                       POP_TOP
                       LOAD_CONST               2 (42)
                       JUMP_FORWARD             L2
               L1:     POP_TOP
                       LOAD_CONST               3 (55)
   
   
 Labels and Jump Targets  Labels and Jump Targets
 -----------------------  -----------------------
   
Line 523 
Line 843 
     >>> c.LOAD_CONST(99)      >>> c.LOAD_CONST(99)
     >>> forward = c.JUMP_IF_FALSE()      >>> forward = c.JUMP_IF_FALSE()
     >>> c( 1, Code.POP_TOP, forward, Return(3) )      >>> c( 1, Code.POP_TOP, forward, Return(3) )
     >>> dis(c.code())      >>> dump(c.code())
       0           0 LOAD_CONST               1 (99)                      LOAD_CONST               1 (99)
                   3 JUMP_IF_FALSE            4 (to 10)                      JUMP_IF_FALSE            L1
                   6 LOAD_CONST               2 (1)                      LOAD_CONST               2 (1)
                   9 POP_TOP                      POP_TOP
             >>   10 LOAD_CONST               3 (3)              L1:     LOAD_CONST               3 (3)
                  13 RETURN_VALUE                      RETURN_VALUE
   
 However, there's an easier way to do the same thing, using ``Label`` objects::  However, there's an easier way to do the same thing, using ``Label`` objects::
   
Line 538 
Line 858 
     >>> skip = Label()      >>> skip = Label()
   
     >>> c(99, skip.JUMP_IF_FALSE, 1, Code.POP_TOP, skip, Return(3))      >>> c(99, skip.JUMP_IF_FALSE, 1, Code.POP_TOP, skip, Return(3))
     >>> dis(c.code())      >>> dump(c.code())
       0           0 LOAD_CONST               1 (99)                      LOAD_CONST               1 (99)
                   3 JUMP_IF_FALSE            4 (to 10)                      JUMP_IF_FALSE            L1
                   6 LOAD_CONST               2 (1)                      LOAD_CONST               2 (1)
                   9 POP_TOP                      POP_TOP
             >>   10 LOAD_CONST               3 (3)              L1:     LOAD_CONST               3 (3)
                  13 RETURN_VALUE                      RETURN_VALUE
   
 This approach has the advantage of being easy to use in complex trees.  This approach has the advantage of being easy to use in complex trees.
 ``Label`` objects have attributes corresponding to every opcode that uses a  ``Label`` objects have attributes corresponding to every opcode that uses a
Line 560 
Line 880 
     AssertionError: Label previously defined      AssertionError: Label previously defined
   
   
   More Conditional Jump Instructions
   ----------------------------------
   
   In Python 2.7, the traditional ``JUMP_IF_TRUE`` and ``JUMP_IF_FALSE``
   instructions were replaced with four new instructions that either conditionally
   or unconditionally pop the value being tested.  This was done to improve
   performance, since virtually all conditional jumps in Python code pop the
   value on one branch or the other.
   
   To provide better cross-version compatibility, BytecodeAssembler emulates the
   old instructions on Python 2.7 by emitting a ``DUP_TOP`` followed by a
   ``POP_JUMP_IF_FALSE`` or ``POP_JUMP_IF_TRUE`` instruction.
   
   However, since this decreases performance, BytecodeAssembler *also* emulates
   Python 2.7's ``JUMP_IF_FALSE_OR_POP`` and ``JUMP_IF_FALSE_OR_TRUE`` opcodes
   on *older* Pythons::
   
       >>> c = Code()
       >>> l1, l2 = Label(), Label()
       >>> c(Local('a'), l1.JUMP_IF_FALSE_OR_POP, Return(27), l1)
       >>> c(l2.JUMP_IF_TRUE_OR_POP, Return(42), l2, Code.RETURN_VALUE)
       >>> dump(c.code())
                       LOAD_FAST                0 (a)
                       JUMP_IF_FALSE           L1
                       POP_TOP
                       LOAD_CONST               1 (27)
                       RETURN_VALUE
               L1:     JUMP_IF_TRUE            L2
                       POP_TOP
                       LOAD_CONST               2 (42)
                       RETURN_VALUE
               L2:     RETURN_VALUE
   
   This means that you can immediately begin using the "or-pop" variations, in
   place of a jump followed by a pop, and BytecodeAssembler will use the faster
   single instruction automatically on Python 2.7+.
   
   BytecodeAssembler *also* supports using Python 2.7's conditional jumps
   that do unconditional pops, but currently cannot emulate them on older Python
   versions, so at the moment you should use them only when your code requires
   Python 2.7.
   
   (Note: for ease in doctesting across Python versions, the ``dump()`` function
   *always* shows the code as if it were generated for Python 2.6 or lower, so
   if you need to check the *actual* bytecodes generated, you must use Python's
   ``dis.dis()`` function instead!)
   
   
   N-Way Comparisons
   -----------------
   
   You can generate N-way comparisons using the ``Compare()`` node type::
   
       >>> from peak.util.assembler import Compare
   
       >>> c = Code()
       >>> c(Compare(Local('a'), [('<', Local('b'))]))
       >>> dis(c.code())
         0           0 LOAD_FAST                0 (a)
                     3 LOAD_FAST                1 (b)
                     6 COMPARE_OP               0 (<)
   
   3-way comparisons generate code that's a bit more complex.  Here's a three-way
   comparison (``a<b<c``)::
   
       >>> c = Code()
       >>> c.return_(Compare(Local('a'), [('<', Local('b')), ('<', Local('c'))]))
       >>> dump(c.code())
                       LOAD_FAST                0 (a)
                       LOAD_FAST                1 (b)
                       DUP_TOP
                       ROT_THREE
                       COMPARE_OP               0 (<)
                       JUMP_IF_FALSE           L1
                       POP_TOP
                       LOAD_FAST                2 (c)
                       COMPARE_OP               0 (<)
                       JUMP_FORWARD            L2
               L1:     ROT_TWO
                       POP_TOP
               L2:     RETURN_VALUE
   
   And a four-way (``a<b>c!=d``)::
   
       >>> c = Code()
       >>> c.return_(
       ...     Compare( Local('a'), [
       ...         ('<', Local('b')), ('>', Local('c')), ('!=', Local('d'))
       ...     ])
       ... )
       >>> dump(c.code())
                       LOAD_FAST                0 (a)
                       LOAD_FAST                1 (b)
                       DUP_TOP
                       ROT_THREE
                       COMPARE_OP               0 (<)
                       JUMP_IF_FALSE           L1
                       POP_TOP
                       LOAD_FAST                2 (c)
                       DUP_TOP
                       ROT_THREE
                       COMPARE_OP               4 (>)
                       JUMP_IF_FALSE           L1
                       POP_TOP
                       LOAD_FAST                3 (d)
                       COMPARE_OP               3 (!=)
                       JUMP_FORWARD            L2
               L1:     ROT_TWO
                       POP_TOP
               L2:     RETURN_VALUE
   
   
   Sequence Unpacking
   ------------------
   
   The ``UnpackSequence`` node type takes a sequence of code generation targets,
   and generates an ``UNPACK_SEQUENCE`` of the correct length, followed by the
   targets::
   
       >>> from peak.util.assembler import UnpackSequence
       >>> c = Code()
       >>> c((1,2), UnpackSequence([LocalAssign('x'), LocalAssign('y')]))
       >>> dis(c.code())   # x, y = 1, 2
         0           0 LOAD_CONST               1 (1)
                     3 LOAD_CONST               2 (2)
                     6 BUILD_TUPLE              2
                     9 UNPACK_SEQUENCE          2
                    12 STORE_FAST               0 (x)
                    15 STORE_FAST               1 (y)
   
   
   Yield Statements
   ----------------
   
   The ``YieldStmt`` node type generates the necessary opcode(s) for a ``yield``
   statement, based on the target Python version.  (In Python 2.5+, a ``POP_TOP``
   must be generated after a ``YIELD_VALUE`` in order to create a yield statement,
   as opposed to a yield expression.)  It also sets the code flags needed to make
   the resulting code object a generator::
   
       >>> from peak.util.assembler import YieldStmt
       >>> c = Code()
       >>> c(YieldStmt(1), YieldStmt(2), Return(None))
       >>> list(eval(c.code()))
       [1, 2]
   
   
   
 Constant Detection and Folding  Constant Detection and Folding
 ==============================  ==============================
   
Line 569 
Line 1037 
   
     >>> from peak.util.assembler import const_value      >>> from peak.util.assembler import const_value
   
     >>> simple_values = [1, 2L, 3.0, 4j+5, "6", u"7", False, None, c.code()]      >>> simple_values = [1, long(2), 3.0, 4j+5, "6", unicode("7"), False, None, c.code()]
   
     >>> map(const_value, simple_values)      >>> list(map(const_value, simple_values))
     [1, 2L, 3.0, (5+4j), '6', u'7', False, None, <code object <lambda> ...>]      [1, 2..., 3.0, (5+4j), '6', ...'7', False, None, <code object <lambda>...>]
   
 Values wrapped in a ``Const()`` are also returned as-is::  Values wrapped in a ``Const()`` are also returned as-is::
   
     >>> map(const_value, map(Const, simple_values))      >>> list(map(const_value, map(Const, simple_values)))
     [1, 2L, 3.0, (5+4j), '6', u'7', False, None, <code object <lambda> ...>]      [1, 2..., 3.0, (5+4j), '6', ...'7', False, None, <code object <lambda>...>]
   
 But no other node types produce constant values; instead, ``NotAConstant`` is  But no other node types produce constant values; instead, ``NotAConstant`` is
 raised::  raised::
Line 585 
Line 1053 
     >>> const_value(Local('x'))      >>> const_value(Local('x'))
     Traceback (most recent call last):      Traceback (most recent call last):
       ...        ...
     NotAConstant: Local('x',)      NotAConstant: Local('x')
   
 Tuples of constants are recursively replaced by constant tuples::  Tuples of constants are recursively replaced by constant tuples::
   
Line 600 
Line 1068 
     >>> const_value( (1,Global('y')) )      >>> const_value( (1,Global('y')) )
     Traceback (most recent call last):      Traceback (most recent call last):
       ...        ...
     NotAConstant: Global('y',)      NotAConstant: Global('y')
   
 As do any types not previously described here::  As do any types not previously described here::
   
Line 626 
Line 1094 
 ``Const`` node instead of a ``Call`` node::  ``Const`` node instead of a ``Call`` node::
   
     >>> Call( Const(type), [1] )      >>> Call( Const(type), [1] )
     Const(<type 'int'>)      Const(<... 'int'>)
   
 Thus, you can also take the ``const_value()`` of such calls::  Thus, you can also take the ``const_value()`` of such calls::
   
Line 637 
Line 1105 
 passed in to another ``Call``::  passed in to another ``Call``::
   
     >>> Call(Const(type), [Call( Const(dict), [], [('x',27)] )])      >>> Call(Const(type), [Call( Const(dict), [], [('x',27)] )])
     Const(<type 'dict'>)      Const(<... 'dict'>)
   
 Notice that this folding takes place eagerly, during AST construction.  If you  Notice that this folding takes place eagerly, during AST construction.  If you
 want to implement delayed folding after constant propagation or variable  want to implement delayed folding after constant propagation or variable
Line 651 
Line 1119 
     >>> c = Code()      >>> c = Code()
     >>> c( Call(Const(type), [1]) )      >>> c( Call(Const(type), [1]) )
     >>> dis(c.code())      >>> dis(c.code())
       0           0 LOAD_CONST               1 (<type 'int'>)        0           0 LOAD_CONST               1 (<... 'int'>)
   
     >>> c = Code()      >>> c = Code()
     >>> c( Call(Const(type), [1], fold=False) )      >>> c( Call(Const(type), [1], fold=False) )
     >>> dis(c.code())      >>> dis(c.code())
       0           0 LOAD_CONST               1 (<type 'type'>)        0           0 LOAD_CONST               1 (<... 'type'>)
                   3 LOAD_CONST               2 (1)                    3 LOAD_CONST               2 (1)
                   6 CALL_FUNCTION            1                    6 CALL_FUNCTION            1
   
Line 675 
Line 1143 
 ``globals()``, in other words.  ``globals()``, in other words.
   
   
   Logical And/Or
   --------------
   
   You can evaluate logical and/or expressions using the ``And`` and ``Or`` node
   types::
   
       >>> from peak.util.assembler import And, Or
   
       >>> c = Code()
       >>> c.return_( And([Local('x'), Local('y')]) )
       >>> dump(c.code())
                       LOAD_FAST                0 (x)
                       JUMP_IF_FALSE           L1
                       POP_TOP
                       LOAD_FAST                1 (y)
               L1:     RETURN_VALUE
   
       >>> c = Code()
       >>> c.return_( Or([Local('x'), Local('y')]) )
       >>> dump(c.code())
                       LOAD_FAST                0 (x)
                       JUMP_IF_TRUE            L1
                       POP_TOP
                       LOAD_FAST                1 (y)
               L1:     RETURN_VALUE
   
   
   True or false constants are folded automatically, avoiding code generation
   for intermediate values that will never be used in the result::
   
       >>> c = Code()
       >>> c.return_( And([1, 2, Local('y')]) )
       >>> dis(c.code())
         0           0 LOAD_FAST                0 (y)
                     3 RETURN_VALUE
   
       >>> c = Code()
       >>> c.return_( And([1, 2, Local('y'), 0]) )
       >>> dump(c.code())
                       LOAD_FAST                0 (y)
                       JUMP_IF_FALSE           L1
                       POP_TOP
                       LOAD_CONST               1 (0)
               L1:     RETURN_VALUE
   
       >>> c = Code()
       >>> c.return_( Or([1, 2, Local('y')]) )
       >>> dis(c.code())
         0           0 LOAD_CONST               1 (1)
                     3 RETURN_VALUE
   
       >>> c = Code()
       >>> c.return_( Or([False, Local('y'), 3]) )
       >>> dump(c.code())
                       LOAD_FAST                0 (y)
                       JUMP_IF_TRUE            L1
                       POP_TOP
                       LOAD_CONST               1 (3)
               L1:     RETURN_VALUE
   
   
 Custom Code Generation  Custom Code Generation
 ======================  ======================
   
Line 746 
Line 1275 
   
     >>> c = Code()      >>> c = Code()
     >>> c( TryFinally(ExprStmt(1), ExprStmt(2)) )      >>> c( TryFinally(ExprStmt(1), ExprStmt(2)) )
     >>> dis(c.code())      >>> dump(c.code())
       0           0 SETUP_FINALLY            8 (to 11)                      SETUP_FINALLY           L1
                   3 LOAD_CONST               1 (1)                      LOAD_CONST               1 (1)
                   6 POP_TOP                      POP_TOP
                   7 POP_BLOCK                      POP_BLOCK
                   8 LOAD_CONST               0 (None)                      LOAD_CONST               0 (None)
             >>   11 LOAD_CONST               2 (2)              L1:     LOAD_CONST               2 (2)
                  14 POP_TOP                      POP_TOP
                  15 END_FINALLY                      END_FINALLY
   
 The ``nodetype()`` decorator is virtually identical to the ``struct()``  The ``nodetype()`` decorator is virtually identical to the ``struct()``
 decorator in the DecoratorTools package, except that it does not support  decorator in the DecoratorTools package, except that it does not support
Line 768 
Line 1297 
   
     >>> tf = TryFinally(ExprStmt(1), ExprStmt(2))      >>> tf = TryFinally(ExprStmt(1), ExprStmt(2))
     >>> tf      >>> tf
     TryFinally(ExprStmt(1,), ExprStmt(2,))      TryFinally(ExprStmt(1), ExprStmt(2))
   
 * It makes named fields accessible::  * It makes named fields accessible::
   
     >>> tf.block1      >>> tf.block1
     ExprStmt(1,)      ExprStmt(1)
   
     >>> tf.block2      >>> tf.block2
     ExprStmt(2,)      ExprStmt(2)
   
 * Hashing and comparison work as expected (handy for algorithms that require  * Hashing and comparison work as expected (handy for algorithms that require
 comparing or caching AST subtrees, such as common subexpression elimination)::    comparing or caching AST subtrees, such as common subexpression
     elimination)::
   
     >>> ExprStmt(1) == ExprStmt(1)      >>> ExprStmt(1) == ExprStmt(1)
     True      True
Line 804 
Line 1334 
   
 If you want to incorporate constant-folding into your AST nodes, you can do  If you want to incorporate constant-folding into your AST nodes, you can do
 so by checking for constant values and folding them at either construction  so by checking for constant values and folding them at either construction
 or code generation time.  For example, this ``And`` node type folds constants  or code generation time.  For example, this ``And`` node type (a simpler
 during code generation, by not generating unnecessary branches when it can  version of the one included in ``peak.util.assembler``) folds constants during
   code generation, by not generating unnecessary branches when it can
 prove which way a branch will go::  prove which way a branch will go::
   
     >>> from peak.util.assembler import NotAConstant      >>> from peak.util.assembler import NotAConstant
Line 819 
Line 1350 
     ...             if const_value(value):      ...             if const_value(value):
     ...                 continue        # true constants can be skipped      ...                 continue        # true constants can be skipped
     ...         except NotAConstant:    # but non-constants require code      ...         except NotAConstant:    # but non-constants require code
     ...             code(value, end.JUMP_IF_FALSE, Code.POP_TOP)      ...             code(value, end.JUMP_IF_FALSE_OR_POP)
     ...         else:       # and false constants end the chain right away      ...         else:       # and false constants end the chain right away
     ...             return code(value, end)      ...             return code(value, end)
     ...     code(values[-1], end)      ...     code(values[-1], end)
Line 839 
Line 1370 
   
     >>> c = Code()      >>> c = Code()
     >>> c.return_( And([Local('x'), False, 27]) )      >>> c.return_( And([Local('x'), False, 27]) )
     >>> dis(c.code())      >>> dump(c.code())
       0           0 LOAD_FAST                0 (x)                      LOAD_FAST                0 (x)
                   3 JUMP_IF_FALSE            4 (to 10)                      JUMP_IF_FALSE           L1
                   6 POP_TOP                      POP_TOP
                   7 LOAD_CONST               1 (False)                      LOAD_CONST               1 (False)
             >>   10 RETURN_VALUE              L1:     RETURN_VALUE
   
 The above example only folds constants at code generation time, however.  You  The above example only folds constants at code generation time, however.  You
 can also do constant folding at AST construction time, using the  can also do constant folding at AST construction time, using the
Line 864 
Line 1395 
     >>> Getattr = nodetype()(Getattr)      >>> Getattr = nodetype()(Getattr)
   
     >>> const_value(Getattr(1, '__class__'))      >>> const_value(Getattr(1, '__class__'))
     <type 'int'>      <... 'int'>
   
 The ``fold_args()`` function tries to evaluate the node immediately, if all of  The ``fold_args()`` function tries to evaluate the node immediately, if all of
 its arguments are constants, by creating a temporary ``Code`` object, and  its arguments are constants, by creating a temporary ``Code`` object, and
Line 883 
Line 1414 
 implements its ``fold`` argument and the suppression of folding when  implements its ``fold`` argument and the suppression of folding when
 the call has no arguments.  the call has no arguments.
   
   (By the way, this same ``Getattr`` node type is also available
   
   
 Setting the Code's Calling Signature  Setting the Code's Calling Signature
 ====================================  ====================================
Line 897 
Line 1430 
     ...     pass      ...     pass
   
     >>> c = Code.from_function(f1)      >>> c = Code.from_function(f1)
     >>> f2 = new.function(c.code(), globals())      >>> f2 = function(c.code(), globals())
   
     >>> import inspect      >>> import inspect
   
     >>> inspect.getargspec(f1)      >>> tuple(inspect.getargspec(f1))
     (['a', 'b'], 'c', 'd', None)      (['a', 'b'], 'c', 'd', None)
   
     >>> inspect.getargspec(f2)      >>> tuple(inspect.getargspec(f2))
     (['a', 'b'], 'c', 'd', None)      (['a', 'b'], 'c', 'd', None)
   
 Note that these constructors do not copy any actual *code* from the code  Note that these constructors do not copy any actual *code* from the code
Line 916 
Line 1449 
     >>> c1 = Code.from_function(f1, copy_lineno=True)      >>> c1 = Code.from_function(f1, copy_lineno=True)
     >>> c1.co_firstlineno      >>> c1.co_firstlineno
     1      1
       >>> c1.co_filename is f1.func_code.co_filename
       True
   
 If you create a ``Code`` instance from a function that has nested positional  If you create a ``Code`` instance from a function that has nested positional
 arguments, the returned code object will include a prologue to unpack the  arguments, the returned code object will include a prologue to unpack the
Line 924 
Line 1459 
     >>> def f3(a, (b,c), (d,(e,f))):      >>> def f3(a, (b,c), (d,(e,f))):
     ...     pass      ...     pass
   
     >>> f4 = new.function(Code.from_function(f3).code(), globals())      >>> f4 = function(Code.from_function(f3).code(), globals())
     >>> dis(f4)      >>> dis(f4)
       0           0 LOAD_FAST                1 (.1)        0           0 LOAD_FAST                1 (.1)
                   3 UNPACK_SEQUENCE          2                    3 UNPACK_SEQUENCE          2
Line 941 
Line 1476 
 unpacking process, and is designed so that the ``inspect`` module will  unpacking process, and is designed so that the ``inspect`` module will
 recognize it as an argument unpacking prologue::  recognize it as an argument unpacking prologue::
   
     >>> inspect.getargspec(f3)      >>> tuple(inspect.getargspec(f3))
     (['a', ['b', 'c'], ['d', ['e', 'f']]], None, None, None)      (['a', ['b', 'c'], ['d', ['e', 'f']]], None, None, None)
   
     >>> inspect.getargspec(f4)      >>> tuple(inspect.getargspec(f4))
     (['a', ['b', 'c'], ['d', ['e', 'f']]], None, None, None)      (['a', ['b', 'c'], ['d', ['e', 'f']]], None, None, None)
   
   You can also use the ``from_spec(name='<lambda>', args=(), var=None, kw=None)``
   classmethod to explicitly set a name and argument spec for a new code object::
   
       >>> c = Code.from_spec('a', ('b', ('c','d'), 'e'), 'f', 'g')
       >>> c.co_name
       'a'
   
       >>> c.co_varnames
       ['b', '.1', 'e', 'f', 'g', 'c', 'd']
   
       >>> c.co_argcount
       3
   
       >>> tuple(inspect.getargs(c.code()))
       (['b', ['c', 'd'], 'e'], 'f', 'g')
   
   
 Code Attributes  Code Attributes
 ===============  ===============
Line 966 
Line 1517 
     >>> c.LOAD_CONST(42)      >>> c.LOAD_CONST(42)
     >>> c.RETURN_VALUE()      >>> c.RETURN_VALUE()
   
     >>> f = new.function(c.code(), globals())      >>> f = function(c.code(), globals())
     >>> f(1,2,3)      >>> f(1,2,3)
     42      42
   
     >>> import inspect      >>> import inspect
     >>> inspect.getargspec(f)      >>> tuple(inspect.getargspec(f))
     (['a', 'b', 'c'], None, None, None)      (['a', 'b', 'c'], None, None, None)
   
 Although Python code objects want ``co_varnames`` to be a tuple, ``Code``  Although Python code objects want ``co_varnames`` to be a tuple, ``Code``
Line 1102 
Line 1653 
   
     >>> c = Code()      >>> c = Code()
     >>> fwd = c.JUMP_FORWARD()      >>> fwd = c.JUMP_FORWARD()
     >>> print c.stack_size  # forward jump marks stack size as unknown      >>> print(c.stack_size)  # forward jump marks stack size as unknown
     None      None
   
     >>> c.LOAD_CONST(42)      >>> c.LOAD_CONST(42)
Line 1124 
Line 1675 
 code that might be unreachable.  For example, consider this ``If``  code that might be unreachable.  For example, consider this ``If``
 implementation::  implementation::
   
     >>> def Pass(code=None):  
     ...     if code is None:  
     ...         return Pass  
   
     >>> def If(cond, then, else_=Pass, code=None):      >>> def If(cond, then, else_=Pass, code=None):
     ...     if code is None:      ...     if code is None:
     ...         return cond, then, else_      ...         return cond, then, else_
     ...     else_clause = Label()      ...     else_clause = Label()
     ...     end_if = Label()      ...     end_if = Label()
     ...     code(cond, else_clause.JUMP_IF_FALSE, Code.POP_TOP, then)      ...     code(cond, else_clause.JUMP_IF_FALSE_OR_POP, then)
     ...     code(end_if.JUMP_FORWARD, else_clause, Code.POP_TOP, else_)      ...     code(end_if.JUMP_FORWARD, else_clause, Code.POP_TOP, else_)
     ...     code(end_if)      ...     code(end_if)
     >>> If = nodetype()(If)      >>> If = nodetype()(If)
Line 1141 
Line 1688 
 It works okay if there's no dead code::  It works okay if there's no dead code::
   
     >>> c = Code()      >>> c = Code()
     >>> c( If(23, 42, 55) )      >>> c( If(Local('a'), 42, 55) )
     >>> dis(c.code())      >>> dump(c.code())
       0           0 LOAD_CONST               1 (23)                      LOAD_FAST                0 (a)
                   3 JUMP_IF_FALSE            7 (to 13)                      JUMP_IF_FALSE           L1
                   6 POP_TOP                      POP_TOP
                   7 LOAD_CONST               2 (42)                      LOAD_CONST               1 (42)
                  10 JUMP_FORWARD             4 (to 17)                      JUMP_FORWARD            L2
             >>   13 POP_TOP              L1:     POP_TOP
                  14 LOAD_CONST               3 (55)                      LOAD_CONST               2 (55)
   
 But it breaks if you end the "then" block with a return::  But it breaks if you end the "then" block with a return::
   
Line 1166 
Line 1713 
     ...         return cond, then, else_      ...         return cond, then, else_
     ...     else_clause = Label()      ...     else_clause = Label()
     ...     end_if = Label()      ...     end_if = Label()
     ...     code(cond, else_clause.JUMP_IF_FALSE, Code.POP_TOP, then)      ...     code(cond, else_clause.JUMP_IF_FALSE_OR_POP, then)
     ...     if code.stack_size is not None:      ...     if code.stack_size is not None:
     ...         end_if.JUMP_FORWARD(code)      ...         end_if.JUMP_FORWARD(code)
     ...     code(else_clause, Code.POP_TOP, else_, end_if)      ...     code(else_clause, Code.POP_TOP, else_, end_if)
Line 1175 
Line 1722 
 As you can see, the dead code is now eliminated::  As you can see, the dead code is now eliminated::
   
     >>> c = Code()      >>> c = Code()
     >>> c( If(23, Return(42), 55) )      >>> c( If(Local('a'), Return(42), 55) )
     >>> dis(c.code())      >>> dump(c.code())
       0           0 LOAD_CONST               1 (23)                      LOAD_FAST                0 (a)
                   3 JUMP_IF_FALSE            5 (to 11)                      JUMP_IF_FALSE           L1
                   6 POP_TOP                      POP_TOP
                   7 LOAD_CONST               2 (42)                      LOAD_CONST               1 (42)
                  10 RETURN_VALUE                      RETURN_VALUE
             >>   11 POP_TOP              L1:     POP_TOP
                  12 LOAD_CONST               3 (55)                      LOAD_CONST               2 (55)
   
   
 Blocks, Loops, and Exception Handling  Blocks, Loops, and Exception Handling
Line 1269 
Line 1816 
     >>> c.POP_TOP()      >>> c.POP_TOP()
     >>> else_()      >>> else_()
     >>> c.return_()      >>> c.return_()
     >>> dis(c.code())      >>> dump(c.code())
       0           0 SETUP_EXCEPT             4 (to 7)                      SETUP_EXCEPT            L1
                   3 POP_BLOCK                      POP_BLOCK
                   4 JUMP_FORWARD             3 (to 10)                      JUMP_FORWARD            L2
             >>    7 POP_TOP              L1:     POP_TOP
                   8 POP_TOP                      POP_TOP
                   9 POP_TOP                      POP_TOP
             >>   10 LOAD_CONST               0 (None)              L2:     LOAD_CONST               0 (None)
                  13 RETURN_VALUE                      RETURN_VALUE
   
 In the example above, an empty block executes with an exception handler that  In the example above, an empty block executes with an exception handler that
 begins at offset 7.  When the block is done, it jumps forward to the end of  begins at offset 7.  When the block is done, it jumps forward to the end of
Line 1298 
Line 1845 
     ...     Return()      ...     Return()
     ... )      ... )
   
     >>> dis(c.code())      >>> dump(c.code())
       0           0 SETUP_EXCEPT             4 (to 7)                      SETUP_EXCEPT             L1
                   3 POP_BLOCK                      POP_BLOCK
                   4 JUMP_FORWARD             3 (to 10)                      JUMP_FORWARD             L2
             >>    7 POP_TOP              L1:     POP_TOP
                   8 POP_TOP                      POP_TOP
                   9 POP_TOP                      POP_TOP
             >>   10 LOAD_CONST               0 (None)              L2:     LOAD_CONST               0 (None)
                  13 RETURN_VALUE                      RETURN_VALUE
   
   (Labels have a ``POP_BLOCK`` attribute that you can pass in when generating
   code.)
   
   And, for generating typical try/except blocks, you can use the ``TryExcept``
   node type, which takes a body, a sequence of exception-type/handler pairs,
   and an optional "else" clause::
   
       >>> from peak.util.assembler import TryExcept
       >>> c = Code()
       >>> c.return_(
       ...     TryExcept(
       ...         Return(1),                                      # body
       ...         [(Const(KeyError),2), (Const(TypeError),3)],    # handlers
       ...         Return(4)                                       # else clause
       ...     )
       ... )
   
 Labels have a ``POP_BLOCK`` attribute that you can pass in when generating      >>> dump(c.code())
 code.                      SETUP_EXCEPT            L1
                       LOAD_CONST               1 (1)
                       RETURN_VALUE
                       POP_BLOCK
                       JUMP_FORWARD            L4
               L1:     DUP_TOP
                       LOAD_CONST               2 (<...KeyError...>)
                       COMPARE_OP              10 (exception match)
                       JUMP_IF_FALSE           L2
                       POP_TOP
                       POP_TOP
                       POP_TOP
                       POP_TOP
                       LOAD_CONST               3 (2)
                       JUMP_FORWARD            L5
               L2:     POP_TOP
                       DUP_TOP
                       LOAD_CONST               4 (<...TypeError...>)
                       COMPARE_OP              10 (exception match)
                       JUMP_IF_FALSE           L3
                       POP_TOP
                       POP_TOP
                       POP_TOP
                       POP_TOP
                       LOAD_CONST               5 (3)
                       JUMP_FORWARD            L5
               L3:     POP_TOP
                       END_FINALLY
               L4:     LOAD_CONST               6 (4)
                       RETURN_VALUE
               L5:     RETURN_VALUE
   
   
 Try/Finally Blocks  Try/Finally Blocks
Line 1329 
Line 1923 
   
 And it produces code that looks like this::  And it produces code that looks like this::
   
     >>> dis(c.code())      >>> dump(c.code())
       0           0 SETUP_FINALLY            4 (to 7)                      SETUP_FINALLY           L1
                   3 POP_BLOCK                      POP_BLOCK
                   4 LOAD_CONST               0 (None)                      LOAD_CONST               0 (None)
             >>    7 END_FINALLY              L1:     END_FINALLY
   
 The ``END_FINALLY`` opcode will remove 1, 2, or 3 values from the stack at  The ``END_FINALLY`` opcode will remove 1, 2, or 3 values from the stack at
 runtime, depending on how the "try" block was exited.  In the case of simply  runtime, depending on how the "try" block was exited.  In the case of simply
Line 1346 
Line 1940 
 adjusts the maximum expected stack size to accomodate up to three values being  adjusts the maximum expected stack size to accomodate up to three values being
 put on the stack by the Python interpreter for exception handling.  put on the stack by the Python interpreter for exception handling.
   
   For your convenience, the ``TryFinally`` node type can also be used to generate
   try/finally blocks::
   
       >>> from peak.util.assembler import TryFinally
       >>> c = Code()
       >>> c( TryFinally(ExprStmt(1), ExprStmt(2)) )
       >>> dump(c.code())
                       SETUP_FINALLY           L1
                       LOAD_CONST               1 (1)
                       POP_TOP
                       POP_BLOCK
                       LOAD_CONST               0 (None)
               L1:     LOAD_CONST               2 (2)
                       POP_TOP
                       END_FINALLY
   
   
 Loops  Loops
 -----  -----
Line 1379 
Line 1989 
     ...     Return()      ...     Return()
     ... )      ... )
   
     >>> dis(c.code())      >>> dump(c.code())
       0           0 SETUP_LOOP              19 (to 22)                      SETUP_LOOP              L3
                   3 LOAD_CONST               1 (5)                      LOAD_CONST               1 (5)
             >>    6 JUMP_IF_FALSE            7 (to 16)              L1:     JUMP_IF_FALSE           L2
                   9 LOAD_CONST               2 (1)                      LOAD_CONST               2 (1)
                  12 BINARY_SUBTRACT                      BINARY_SUBTRACT
                  13 JUMP_ABSOLUTE            6                      JUMP_ABSOLUTE           L1
             >>   16 POP_TOP              L2:     POP_TOP
                  17 POP_BLOCK                      POP_BLOCK
                  18 LOAD_CONST               3 (42)                      LOAD_CONST               3 (42)
                  21 RETURN_VALUE                      RETURN_VALUE
             >>   22 LOAD_CONST               0 (None)              L3:     LOAD_CONST               0 (None)
                  25 RETURN_VALUE                      RETURN_VALUE
   
     >>> eval(c.code())      >>> eval(c.code())
     42      42
Line 1424 
Line 2034 
     >>> fwd()      >>> fwd()
     >>> c.BREAK_LOOP()      >>> c.BREAK_LOOP()
     >>> c.POP_BLOCK()()      >>> c.POP_BLOCK()()
     >>> dis(c.code())      >>> dump(c.code())
       0           0 LOAD_CONST               1 (57)                      LOAD_CONST               1 (57)
                   3 SETUP_LOOP               8 (to 14)                      SETUP_LOOP              L3
                   6 JUMP_IF_TRUE             3 (to 12)                      JUMP_IF_TRUE            L2
             >>    9 JUMP_ABSOLUTE            9              L1:     JUMP_ABSOLUTE           L1
             >>   12 BREAK_LOOP              L2:     BREAK_LOOP
                  13 POP_BLOCK                      POP_BLOCK
   
 In other words, ``CONTINUE_LOOP`` only really emits a ``CONTINUE_LOOP`` opcode  In other words, ``CONTINUE_LOOP`` only really emits a ``CONTINUE_LOOP`` opcode
 if it's inside some other kind of block within the loop, e.g. a "try" clause::  if it's inside some other kind of block within the loop, e.g. a "try" clause::
Line 1446 
Line 2056 
     >>> c.POP_BLOCK()      >>> c.POP_BLOCK()
     >>> c.END_FINALLY()      >>> c.END_FINALLY()
     >>> c.POP_BLOCK()()      >>> c.POP_BLOCK()()
       >>> dump(c.code())
                       LOAD_CONST               1 (57)
                       SETUP_LOOP              L4
               L1:     SETUP_FINALLY           L3
                       JUMP_IF_TRUE            L2
                       CONTINUE_LOOP           L1
               L2:     POP_BLOCK
                       LOAD_CONST               0 (None)
               L3:     END_FINALLY
                       POP_BLOCK
   
   ``for`` Loops
   -------------
   
   There is a ``For()`` node type available for generating simple loops (without
   break/continue support).  It takes an iterable expression, an assignment
   clause, and a loop body::
   
       >>> from peak.util.assembler import For
       >>> y = Call(Const(list), (Call(Const(range), (3,)),))
       >>> x = LocalAssign('x')
       >>> body = Suite([Local('x'), Code.PRINT_EXPR])
   
       >>> c = Code()
       >>> c(For(y, x, body))  # for x in range(3): print x
       >>> c.return_()
       >>> dump(c.code())
                       LOAD_CONST               1 ([0, 1, 2])
                       GET_ITER
               L1:     FOR_ITER                L2
                       STORE_FAST               0 (x)
                       LOAD_FAST                0 (x)
                       PRINT_EXPR
                       JUMP_ABSOLUTE           L1
               L2:     LOAD_CONST               0 (None)
                       RETURN_VALUE
   
   The arguments are given in execution order: first the "in" value of the loop,
   then the assignment to a loop variable, and finally the body of the loop.  The
   distinction between the assignment and body, however, is only for clarity and
   convenience (to avoid needing to glue the assignment to the body with a
   ``Suite``).  If you already have a suite or only need one node for the entire
   loop body, you can do the same thing with only two arguments::
   
       >>> c = Code()
       >>> c(For(y, Code.PRINT_EXPR))
       >>> c.return_()
       >>> dump(c.code())
                       LOAD_CONST               1 ([0, 1, 2])
                       GET_ITER
               L1:     FOR_ITER                L2
                       PRINT_EXPR
                       JUMP_ABSOLUTE           L1
               L2:     LOAD_CONST               0 (None)
                       RETURN_VALUE
   
   Notice, by the way, that ``For()`` does NOT set up a loop block for you, so if
   you want to be able to use break and continue, you'll need to wrap the loop in
   a labelled SETUP_LOOP/POP_BLOCK pair, as described in the preceding sections.
   
   
   List Comprehensions
   -------------------
   
   In order to generate correct list comprehension code for the target Python
   version, you must use the ``ListComp()`` and ``LCAppend()`` node types.  This
   is because Python versions 2.4 and up store the list being built in a temporary
   variable, and use a special ``LIST_APPEND`` opcode to append values, while 2.3
   stores the list's ``append()`` method in the temporary variable, and calls it
   to append values.
   
   The ``ListComp()`` node wraps a code body (usually a ``For()`` loop) and
   manages the creation and destruction of a temporary variable (e.g. ``_[1]``,
   ``_[2]``, etc.).  The ``LCAppend()`` node type wraps a value or expression to
   be appended to the innermost active ``ListComp()`` in progress::
   
       >>> from peak.util.assembler import ListComp, LCAppend
       >>> c = Code()
       >>> simple = ListComp(For(y, x, LCAppend(Local('x'))))
       >>> c.return_(simple)
       >>> eval(c.code())
       [0, 1, 2]
   
       >>> c = Code()
       >>> c.return_(ListComp(For(y, x, LCAppend(simple))))
       >>> eval(c.code())
       [[0, 1, 2], [0, 1, 2], [0, 1, 2]]
   
   
   Closures and Nested Functions
   =============================
   
   Free and Cell Variables
   -----------------------
   
   To implement closures and nested scopes, your code objects must use "free" or
   "cell" variables in place of regular "fast locals".  A "free" variable is one
   that is defined in an outer scope, and a "cell" variable is one that's defined
   in the current scope, but will also be used by nested functions.
   
   The simplest way to set up free or cell variables is to use a code object's
   ``makefree(names)`` and ``makecells(names)`` methods::
   
       >>> c = Code()
       >>> c.co_cellvars
       ()
       >>> c.co_freevars
       ()
   
       >>> c.makefree(['x', 'y'])
       >>> c.makecells(['z'])
   
       >>> c.co_cellvars
       ('z',)
       >>> c.co_freevars
       ('x', 'y')
   
   When a name has been defined as a free or cell variable, the ``_DEREF`` opcode
   variants are used to generate ``Local()`` and ``LocalAssign()`` nodes::
   
       >>> c((Local('x'), Local('y')), LocalAssign('z'))
       >>> dis(c.code())
         0           0 LOAD_DEREF               1 (x)
                     3 LOAD_DEREF               2 (y)
                     6 BUILD_TUPLE              2
                     9 STORE_DEREF              0 (z)
   
   If you have already written code in a code object that operates on the relevant
   locals, the code is retroactively patched to use the ``_DEREF`` opcodes::
   
       >>> c = Code()
       >>> c((Local('x'), Local('y')), LocalAssign('z'))
     >>> dis(c.code())      >>> dis(c.code())
       0           0 LOAD_CONST               1 (57)        0           0 LOAD_FAST                0 (x)
                   3 SETUP_LOOP              15 (to 21)                    3 LOAD_FAST                1 (y)
             >>    6 SETUP_FINALLY           10 (to 19)                    6 BUILD_TUPLE              2
                   9 JUMP_IF_TRUE             3 (to 15)                    9 STORE_FAST               2 (z)
                  12 CONTINUE_LOOP            6  
             >>   15 POP_BLOCK      >>> c.makefree(['x', 'y'])
                  16 LOAD_CONST               0 (None)      >>> c.makecells(['z'])
             >>   19 END_FINALLY  
                  20 POP_BLOCK      >>> dis(c.code())
         0           0 LOAD_DEREF               1 (x)
                     3 LOAD_DEREF               2 (y)
                     6 BUILD_TUPLE              2
                     9 STORE_DEREF              0 (z)
   
   This means that you can defer the decision of which locals are free/cell
   variables until the code is ready to be generated.  In fact, by passing in
   a "parent" code object to the ``.code()`` method, you can get BytecodeAssembler
   to automatically call ``makefree()`` and ``makecells()`` for the correct
   variable names in the child and parent code objects, as we'll see in the next
   section.
   
   
   Nested Code Objects
   -------------------
   
   To create a code object for use in a nested scope, you can use the parent code
   object's ``nested()`` method.  It works just like the ``from_spec()``
   classmethod, except that the ``co_filename`` of the parent is copied to the
   child::
   
       >>> p = Code()
       >>> p.co_filename = 'testname'
   
       >>> c = p.nested('sub', ['a','b'], 'c', 'd')
   
       >>> c.co_name
       'sub'
   
       >>> c.co_filename
       'testname'
   
       >>> tuple(inspect.getargs(c.code(p)))
       (['a', 'b'], 'c', 'd')
   
   Notice that you must pass the parent code object to the child's ``.code()``
   method to ensure that free/cell variables are properly set up.  When the
   ``code()`` method is given another code object as a parameter, it automatically
   converts any locally-read (but not written) to "free" variables in the child
   code, and ensures that those same variables become "cell" variables in the
   supplied parent code object::
   
       >>> p.LOAD_CONST(42)
       >>> p(LocalAssign('a'))
       >>> dis(p.code())
         0           0 LOAD_CONST               1 (42)
                     3 STORE_FAST               0 (a)
   
       >>> c = p.nested()
       >>> c(Local('a'))
   
       >>> dis(c.code(p))
         0           0 LOAD_DEREF               0 (a)
   
       >>> dis(p.code())
         0           0 LOAD_CONST               1 (42)
                     3 STORE_DEREF              0 (a)
   
   Notice that the ``STORE_FAST`` in the parent code object was automatically
   patched to a ``STORE_DEREF``, with an updated offset if applicable.  Any
   future use of ``Local('a')`` or ``LocalAssign('a')`` in the parent or child
   code objects will now refer to the free/cell variable, rather than the "local"
   variable::
   
       >>> p(Local('a'))
       >>> dis(p.code())
         0           0 LOAD_CONST               1 (42)
                     3 STORE_DEREF              0 (a)
                     6 LOAD_DEREF               0 (a)
   
       >>> c(LocalAssign('a'))
       >>> dis(c.code(p))
         0           0 LOAD_DEREF               0 (a)
                     3 STORE_DEREF              0 (a)
   
   
   ``Function()``
   --------------
   
   The ``Function(body, name='<lambda>', args=(), var=None, kw=None, defaults=())``
   node type creates a function object from the specified body and the optional
   name, argument specs, and defaults.  The ``Function()`` node generates code to
   create the function object with the appropriate defaults and closure (if
   applicable), and any needed free/cell variables are automatically set up in the
   parent and child code objects.  The newly generated function will be on top of
   the stack at the end of the generated code::
   
       >>> from peak.util.assembler import Function
       >>> c = Code()
       >>> c.co_filename = '<string>'
       >>> c.return_(Function(Return(Local('a')), 'f', ['a'], defaults=[42]))
       >>> dis(c.code())
         0           0 LOAD_CONST               1 (42)
                     3 LOAD_CONST               2 (<... f..., file ...<string>..., line ...>)
                     6 MAKE_FUNCTION            1
                     9 RETURN_VALUE
   
   Now that we've generated the code for a function returning a function, let's
   run it, to get the function we defined::
   
       >>> f = eval(c.code())
       >>> f
       <function f at ...>
   
       >>> tuple(inspect.getargspec(f))
       (['a'], None, None, (42,))
   
       >>> f()
       42
   
       >>> f(99)
       99
   
   Now let's create a doubly nested function, with some extras::
   
       >>> c = Code()
       >>> c.co_filename = '<string>'
       >>> c.return_(
       ...     Function(Return(Function(Return(Local('a')))),
       ...     'f', ['a', 'b'], 'c', 'd', [99, 66])
       ... )
       >>> dis(c.code())
         0           0 LOAD_CONST               1 (99)
                     3 LOAD_CONST               2 (66)
                     6 LOAD_CONST               3 (<... f..., file ...<string>..., line ...>)
                     9 MAKE_FUNCTION            2
                    12 RETURN_VALUE
   
       >>> f = eval(c.code())
       >>> f
       <function f at ...>
   
       >>> tuple(inspect.getargspec(f))
       (['a', 'b'], 'c', 'd', (99, 66))
   
       >>> dis(f)
         0           0 LOAD_CLOSURE             0 (a)
                     ... LOAD_CONST               1 (<... <lambda>..., file ...<string>..., line ...>)
                     ... MAKE_CLOSURE             0
                     ... RETURN_VALUE
   
       >>> dis(f())
         0           0 LOAD_DEREF               0 (a)
                     3 RETURN_VALUE
   
       >>> f(42)()
       42
   
       >>> f()()
       99
   
   As you can see, ``Function()`` not only takes care of setting up free/cell
   variables in all the relevant scopes, it also chooses whether to use
   ``MAKE_FUNCTION`` or ``MAKE_CLOSURE``, and generates code for the defaults.
   
   (Note, by the way, that the `defaults` argument should be a sequence of
   generatable expressions; in the examples here, we used numbers, but they could
   have been arbitrary expression nodes.)
   
   
 ----------------------  ----------------------
Line 1479 
Line 2379 
     >>> simple_code(1,1).co_stacksize      >>> simple_code(1,1).co_stacksize
     1      1
   
     >>> dis(simple_code(13,414))    # FAILURE EXPECTED IN PYTHON 2.3      >>> dis(simple_code(13,414))
      13           0 LOAD_CONST               0 (None)       13           0 LOAD_CONST               0 (None)
     414           3 RETURN_VALUE      414           3 RETURN_VALUE
   
Line 1492 
Line 2392 
     >>> simple_code(13,14,100).co_stacksize      >>> simple_code(13,14,100).co_stacksize
     100      100
   
     >>> dis(simple_code(13,572,120))    # FAILURE EXPECTED IN Python 2.3      >>> dis(simple_code(13,572,120))
      13           0 LOAD_CONST               0 (None)       13           0 LOAD_CONST               0 (None)
                   3 LOAD_CONST               0 (None)                    3 LOAD_CONST               0 (None)
     ...      ...
Line 1551 
Line 2451 
                   3 LOAD_ATTR                1 (bar)                    3 LOAD_ATTR                1 (bar)
                   6 DELETE_FAST              0 (baz)                    6 DELETE_FAST              0 (baz)
   
   Code iteration::
   
       >>> c.DUP_TOP()
       >>> c.return_(Code.POP_TOP)
       >>> list(c) == [
       ...     (0, op.LOAD_GLOBAL, 0),
       ...     (3, op.LOAD_ATTR, 1),
       ...     (6, op.DELETE_FAST, 0),
       ...     (9, op.DUP_TOP, None),
       ...     (10, op.POP_TOP, None),
       ...     (11, op.RETURN_VALUE, None)
       ... ]
       True
   
   Code patching::
   
       >>> c = Code()
       >>> c.LOAD_CONST(42)
       >>> c.STORE_FAST('x')
       >>> c.LOAD_FAST('x')
       >>> c.DELETE_FAST('x')
       >>> c.RETURN_VALUE()
   
       >>> dis(c.code())
         0           0 LOAD_CONST               1 (42)
                     3 STORE_FAST               0 (x)
                     6 LOAD_FAST                0 (x)
                     9 DELETE_FAST              0 (x)
                    12 RETURN_VALUE
   
   
       >>> c.co_varnames
       ['x']
       >>> c.co_varnames.append('y')
   
       >>> c._patch(
       ...     {op.LOAD_FAST:  op.LOAD_FAST,
       ...      op.STORE_FAST: op.STORE_FAST,
       ...      op.DELETE_FAST: op.DELETE_FAST},
       ...     {0: 1}
       ... )
   
       >>> dis(c.code())
         0           0 LOAD_CONST               1 (42)
                     3 STORE_FAST               1 (y)
                     6 LOAD_FAST                1 (y)
                     9 DELETE_FAST              1 (y)
                    12 RETURN_VALUE
   
       >>> c._patch({op.RETURN_VALUE: op.POP_TOP})
       >>> dis(c.code())
         0           0 LOAD_CONST               1 (42)
                     3 STORE_FAST               1 (y)
                     6 LOAD_FAST                1 (y)
                     9 DELETE_FAST              1 (y)
                    12 POP_TOP
   
   Converting locals to free/cell vars::
   
       >>> c = Code()
       >>> c.LOAD_CONST(42)
       >>> c.STORE_FAST('x')
       >>> c.LOAD_FAST('x')
   
       >>> dis(c.code())
         0           0 LOAD_CONST               1 (42)
                     3 STORE_FAST               0 (x)
                     6 LOAD_FAST                0 (x)
   
       >>> c.co_freevars = 'y', 'x'
       >>> c.co_cellvars = 'z',
   
       >>> c._locals_to_cells()
       >>> dis(c.code())
         0           0 LOAD_CONST               1 (42)
                     3 STORE_DEREF              2 (x)
                     6 LOAD_DEREF               2 (x)
   
       >>> c.DELETE_FAST('x')
       >>> c._locals_to_cells()
       Traceback (most recent call last):
         ...
       AssertionError: Can't delete local 'x' used in nested scope
   
       >>> c = Code()
       >>> c.LOAD_CONST(42)
       >>> c.STORE_FAST('x')
       >>> c.LOAD_FAST('x')
   
       >>> c.co_freevars
       ()
       >>> c.makefree(['x'])
       >>> c.co_freevars
       ('x',)
   
       >>> dis(c.code())
         0           0 LOAD_CONST               1 (42)
                     3 STORE_DEREF              0 (x)
                     6 LOAD_DEREF               0 (x)
   
       >>> c = Code()
       >>> c.LOAD_CONST(42)
       >>> c.STORE_FAST('x')
       >>> c.LOAD_FAST('x')
       >>> c.makecells(['x'])
       >>> c.co_freevars
       ()
       >>> c.co_cellvars
       ('x',)
       >>> dis(c.code())
         0           0 LOAD_CONST               1 (42)
                     3 STORE_DEREF              0 (x)
                     6 LOAD_DEREF               0 (x)
   
       >>> c = Code()
       >>> c.LOAD_CONST(42)
       >>> c.STORE_FAST('x')
       >>> c.LOAD_FAST('x')
       >>> c.makefree('x')
       >>> c.makecells(['y'])
       >>> c.co_freevars
       ('x',)
       >>> c.co_cellvars
       ('y',)
       >>> dis(c.code())
         0           0 LOAD_CONST               1 (42)
                     3 STORE_DEREF              1 (x)
                     6 LOAD_DEREF               1 (x)
   
       >>> c = Code()
       >>> c.co_flags &= ~op.CO_OPTIMIZED
       >>> c.makecells(['q'])
       Traceback (most recent call last):
         ...
       AssertionError: Can't use cellvars in unoptimized scope
   
   
   
   Auto-free promotion with code parent:
   
       >>> p = Code()
       >>> c = Code()
       >>> c.LOAD_FAST('x')
       >>> dis(c.code(p))
         0           0 LOAD_DEREF               0 (x)
       >>> p.co_cellvars
       ('x',)
   
       >>> p = Code()
       >>> c = Code.from_function(lambda x,y,z=2: None)
       >>> c.LOAD_FAST('x')
       >>> c.LOAD_FAST('y')
       >>> c.LOAD_FAST('z')
   
       >>> dis(c.code(p))
         0           0 LOAD_FAST                0 (x)
                     3 LOAD_FAST                1 (y)
                     6 LOAD_FAST                2 (z)
       >>> p.co_cellvars
       ()
   
       >>> c.LOAD_FAST('q')
       >>> dis(c.code(p))
         0           0 LOAD_FAST                0 (x)
                     3 LOAD_FAST                1 (y)
                     6 LOAD_FAST                2 (z)
                     9 LOAD_DEREF               0 (q)
       >>> p.co_cellvars
       ('q',)
   
       >>> p = Code()
       >>> c = Code.from_function(lambda x,*y,**z: None)
       >>> c.LOAD_FAST('q')
       >>> c.LOAD_FAST('x')
       >>> c.LOAD_FAST('y')
       >>> c.LOAD_FAST('z')
       >>> dis(c.code(p))
         0           0 LOAD_DEREF               0 (q)
                     3 LOAD_FAST                0 (x)
                     6 LOAD_FAST                1 (y)
                     9 LOAD_FAST                2 (z)
       >>> p.co_cellvars
       ('q',)
   
       >>> p = Code()
       >>> c = Code.from_function(lambda x,*y: None)
       >>> c.LOAD_FAST('x')
       >>> c.LOAD_FAST('y')
       >>> c.LOAD_FAST('z')
       >>> dis(c.code(p))
         0           0 LOAD_FAST                0 (x)
                     3 LOAD_FAST                1 (y)
                     6 LOAD_DEREF               0 (z)
       >>> p.co_cellvars
       ('z',)
   
       >>> p = Code()
       >>> c = Code.from_function(lambda x,**y: None)
       >>> c.LOAD_FAST('x')
       >>> c.LOAD_FAST('y')
       >>> c.LOAD_FAST('z')
       >>> dis(c.code(p))
         0           0 LOAD_FAST                0 (x)
                     3 LOAD_FAST                1 (y)
                     6 LOAD_DEREF               0 (z)
       >>> p.co_cellvars
       ('z',)
   
   
 Stack tracking on jumps::  Stack tracking on jumps::
   
     >>> c = Code()      >>> c = Code()
     >>> else_ = Label()      >>> else_ = Label()
     >>> end = Label()      >>> end = Label()
     >>> c(99, else_.JUMP_IF_TRUE, Code.POP_TOP, end.JUMP_FORWARD)      >>> c(99, else_.JUMP_IF_TRUE_OR_POP, end.JUMP_FORWARD)
     >>> c(else_, Code.POP_TOP, end)      >>> c(else_, Code.POP_TOP, end)
     >>> dis(c.code())      >>> dump(c.code())
       0           0 LOAD_CONST               1 (99)                      LOAD_CONST               1 (99)
                   3 JUMP_IF_TRUE             4 (to 10)                      JUMP_IF_TRUE            L1
                   6 POP_TOP                      POP_TOP
                   7 JUMP_FORWARD             1 (to 11)                      JUMP_FORWARD            L2
             >>   10 POP_TOP              L1:     POP_TOP
   
     >>> c.stack_size      >>> c.stack_size
     0      0
     >>> c.stack_history      >>> if sys.version>='2.7':
     [0, 1, 1, 1, 1, 1, 1, 0, None, None, 1]      ...     print(c.stack_history == [0, 1, 1, 1,    0, 0, 0, None, None, 1])
       ... else:
       ...     print(c.stack_history == [0, 1, 1, 1, 1, 1, 1, 0, None, None, 1])
       True
   
   
     >>> c = Code()      >>> c = Code()
     >>> fwd = c.JUMP_FORWARD()      >>> fwd = c.JUMP_FORWARD()
Line 1585 
Line 2697 
       ...        ...
     AssertionError: Stack level mismatch: actual=1 expected=0      AssertionError: Stack level mismatch: actual=1 expected=0
   
       >>> from peak.util.assembler import For
       >>> c = Code()
       >>> c(For((), Code.POP_TOP, Pass))
       >>> c.return_()
       >>> dump(c.code())
                       BUILD_TUPLE              0
                       GET_ITER
               L1:     FOR_ITER                L2
                       POP_TOP
                       JUMP_ABSOLUTE           L1
               L2:     LOAD_CONST               0 (None)
                       RETURN_VALUE
   
       >>> c.stack_history
       [0, 1, 1, 1, 1, 2, 2, 2, 1, None, None, 0, 1, 1, 1]
   
   
   Yield value::
   
       >>> import sys
       >>> from peak.util.assembler import CO_GENERATOR
       >>> c = Code()
       >>> c.co_flags & CO_GENERATOR
       0
       >>> c(42, Code.YIELD_VALUE)
       >>> c.stack_size == int(sys.version>='2.5')
       True
       >>> (c.co_flags & CO_GENERATOR) == CO_GENERATOR
       True
   
   
   
Line 1745 
Line 2885 
       ...        ...
     AssertionError: Stack underflow      AssertionError: Stack underflow
   
     >>> c.LOAD_CONST(1)      >>> c = Code()
     >>> c.LOAD_CONST(2) # simulate being a function      >>> c.LOAD_CONST(1) # closure
     >>> c.MAKE_CLOSURE(1, 0)      >>> if sys.version>='2.5': c.BUILD_TUPLE(1)
       >>> c.LOAD_CONST(2) # default
       >>> c.LOAD_CONST(3) # simulate being a function
       >>> c.MAKE_CLOSURE(1, 1)
     >>> c.stack_size      >>> c.stack_size
     1      1
   
     >>> c = Code()      >>> c = Code()
     >>> c.LOAD_CONST(1)      >>> c.LOAD_CONST(1)
     >>> c.LOAD_CONST(2)      >>> c.LOAD_CONST(2)
       >>> if sys.version>='2.5': c.BUILD_TUPLE(2)
     >>> c.LOAD_CONST(3) # simulate being a function      >>> c.LOAD_CONST(3) # simulate being a function
     >>> c.MAKE_CLOSURE(1, 1)      >>> c.MAKE_CLOSURE(0, 2)
     >>> c.stack_size      >>> c.stack_size
     1      1
   
   
   
 Labels and backpatching forward references::  Labels and backpatching forward references::
   
     >>> c = Code()      >>> c = Code()
     >>> where = c.here()      >>> where = c.here()
     >>> c.LOAD_CONST(1)      >>> c.LOAD_CONST(1)
     >>> c.JUMP_IF_TRUE(where)      >>> c.JUMP_FORWARD(where)
     Traceback (most recent call last):      Traceback (most recent call last):
       ...        ...
     AssertionError: Relative jumps can't go backwards      AssertionError: Relative jumps can't go backwards
Line 1830 
Line 2975 
     >>> c = Code()      >>> c = Code()
     >>> c.return_(Call(Const(type), [], [], (1,)))      >>> c.return_(Call(Const(type), [], [], (1,)))
     >>> dis(c.code())      >>> dis(c.code())
       0           0 LOAD_CONST               1 (<type 'int'>)        0           0 LOAD_CONST               1 (<... 'int'>)
                   3 RETURN_VALUE                    3 RETURN_VALUE
   
   
Line 1840 
Line 2985 
       0           0 LOAD_CONST               1 ({'x': 1})        0           0 LOAD_CONST               1 ({'x': 1})
                   3 RETURN_VALUE                    3 RETURN_VALUE
   
   Try/Except stack level tracking::
   
       >>> def class_or_type_of(expr):
       ...     return Suite([expr, TryExcept(
       ...         Suite([Getattr(Code.DUP_TOP, '__class__'), Code.ROT_TWO]),
       ...         [(Const(AttributeError), Call(Const(type), (Code.ROT_TWO,)))]
       ...     )])
   
       >>> def type_or_class(x): pass
       >>> c = Code.from_function(type_or_class)
       >>> c.return_(class_or_type_of(Local('x')))
       >>> dump(c.code())
                       LOAD_FAST                0 (x)
                       SETUP_EXCEPT            L1
                       DUP_TOP
                       LOAD_ATTR                0 (__class__)
                       ROT_TWO
                       POP_BLOCK
                       JUMP_FORWARD            L3
               L1:     DUP_TOP
                       LOAD_CONST               1 (<...AttributeError...>)
                       COMPARE_OP              10 (exception match)
                       JUMP_IF_FALSE           L2
                       POP_TOP
                       POP_TOP
                       POP_TOP
                       POP_TOP
                       LOAD_CONST               2 (<... 'type'>)
                       ROT_TWO
                       CALL_FUNCTION            1
                       JUMP_FORWARD            L3
               L2:     POP_TOP
                       END_FINALLY
               L3:     RETURN_VALUE
   
       >>> type_or_class.__code__ = type_or_class.func_code = c.code()
       >>> type_or_class(23)
       <... 'int'>
   
   
   
   
   
 Demo: "Computed Goto"/"Switch Statement"  Demo: "Computed Goto"/"Switch Statement"
 ========================================  ========================================
Line 1849 
Line 3036 
   
     >>> from peak.util.assembler import LOAD_CONST, POP_BLOCK      >>> from peak.util.assembler import LOAD_CONST, POP_BLOCK
   
     >>> def Pass(code=None):  
     ...     if code is None:  
     ...         return Pass  
   
     >>> import sys      >>> import sys
     >>> WHY_CONTINUE = {'2.3':5, '2.4':32, '2.5':32}[sys.version[:3]]      >>> WHY_CONTINUE = {'2.3':5}.get(sys.version[:3], 32)
   
     >>> def Switch(expr, cases, default=Pass, code=None):      >>> def Switch(expr, cases, default=Pass, code=None):
     ...     if code is None:      ...     if code is None:
Line 1894 
Line 3077 
     >>> c(Switch(Local('x'), [(1,Return(42)),(2,Return("foo"))], Return(27)))      >>> c(Switch(Local('x'), [(1,Return(42)),(2,Return("foo"))], Return(27)))
     >>> c.return_()      >>> c.return_()
   
     >>> f = new.function(c.code(), globals())      >>> f = function(c.code(), globals())
     >>> f(1)      >>> f(1)
     42      42
     >>> f(2)      >>> f(2)
Line 1902 
Line 3085 
     >>> f(3)      >>> f(3)
     27      27
   
     >>> dis(c.code())      >>> dump(c.code())
       0           0 SETUP_LOOP              30 (to 33)                      SETUP_LOOP              L2
                   3 LOAD_CONST               1 (<...method get of dict...>)                      LOAD_CONST               1 (<...method ...get of ...>)
                   6 LOAD_FAST                0 (x)                      LOAD_FAST                0 (x)
                   9 CALL_FUNCTION            1                      CALL_FUNCTION            1
                  12 JUMP_IF_FALSE           12 (to 27)                      JUMP_IF_FALSE           L1
                  15 LOAD_CONST               2 (...)                      LOAD_CONST               2 (...)
                  18 END_FINALLY                      END_FINALLY
                  19 LOAD_CONST               3 (42)                      LOAD_CONST               3 (42)
                  22 RETURN_VALUE                      RETURN_VALUE
                  23 LOAD_CONST               4 ('foo')                      LOAD_CONST               4 ('foo')
                  26 RETURN_VALUE                      RETURN_VALUE
             >>   27 POP_TOP              L1:     POP_TOP
                  28 LOAD_CONST               5 (27)                      LOAD_CONST               5 (27)
                  31 RETURN_VALUE                      RETURN_VALUE
                  32 POP_BLOCK                      POP_BLOCK
             >>   33 LOAD_CONST               0 (None)              L2:     LOAD_CONST               0 (None)
                  36 RETURN_VALUE                      RETURN_VALUE
   
   
 TODO  TODO
Line 1931 
Line 3114 
   
 * Exhaustive tests of all opcodes' stack history effects  * Exhaustive tests of all opcodes' stack history effects
   
 * YIELD_EXPR should set CO_GENERATOR; stack effects depend on Python version  * Test wide jumps and wide argument generation in general
   
   * Remove/renumber local variables when a local is converted to free/cell
   

Generate output suitable for use with a patch program
Legend:
Removed from v.2209  
changed lines
  Added in v.2732
cvs-admin@eby-sarna.com

Powered by ViewCVS 1.0-dev

ViewCVS and CVS Help