Log of /PEAK/TODO.txt

Parent Directory

No default branch
Bookmark a link to HEAD: (view) (download)

Added a ZConfig schema for 'running.commands.EventDriven' applications,
a ZConfig component definition for adaptive tasks, and a running shortcut
called 'EventDriven'.  It should now be possible to do this::

     #!/usr/bin/env peak EventDriven

at the top of a ZConfig file formatted according to the new schema, and
have it run.  Unfortunately, so far the only type of task that can be
included is 'running.daemons.AdaptiveTask', which doesn't actually *do*
anything.  So, right now doing this is equivalent to creating an over-
engineered 'sleep' command.  ;)

Also, fixed a problem in ZConfig 'schema.dtd'; I used 'PCDATA' where I
should've used 'CDATA', and I fixed various problems with the
'fromZConfig()' component constructor.  It's kind of annoying that ZConfig
defaults unspecified items to 'None', even if you said they weren't
required and had no default. :(  I'll see if I can do something about this
later; for now I just delete 'None' values from the data.

Added ZConfig URL support.  ZConfig schemas and files can be loaded from
any PEAK-supported stream URL, including 'pkgfile:'.  Interfile references
aren't tested yet, nor are there any unit tests.  Those will be added once
we have some meaningful schemas and configurations to test with.  There
is a 'zconfig.schema' URL scheme that loads a command interpreter, and
a 'peak ZConfig' option as well.  See 'CHANGES.txt' for details on the new
features.  Also, fixed up some issues with the 'fromZConfig()' component
constructor.  All we need now are some useful base schemas for daemons,
and the last piece for 0.5a2 (besides PyProtocols) will be in place.

Added the UML 1.4 metamodel, and thus the ability to load UML 1.4
models encoded in XMI 1.1.

Added support in the mof2py code generator for "unprefixing" enumerated
values, so that UML and other metamodels' enumerations work correctly
when loading from XMI.

Fixed source distributions missing essential setup files.

Rescheduled UML 1.5 and CWM support to version 0.6.  Hardly any tools
use UML 1.5 yet, and we can't do anything useful with CWM until we have
XMI writing, and some type of UI for editing models.

Began the move of 'peak.interface' to a separate 'protocols' package for
ease of distribution independently of PEAK.  Also, removed caching of
success/failed adapter lookups, because there was no way for them to be
cleared when declarations were made for base classes of a class whose
lookup was cached.  (To do caching, I'll have to add a separate cache.)

In the next pass, I'll add the new declaration API that Ty and I designed,
and then change PEAK to use it, factoring out the interim API, and changing
the few remaining introspections in PEAK to work via adapation.  Then, it's
off to documentation, unit tests, and packaging for the 'protocols'
package.

Farewell, 'zope.interface'.  You served us well.

Misc. doc edits

Updated TODO.

Added 'fcgiapp' to distribution; this should be the final checkin for
0.5a1, barring any bug or doc fixes that come up while prepping
the release.

Paramegeddon!  Adjusted API signatures so that all calls that have a
context component, have it as the first parameter.  Changed functions,
methods, and classes are:

* binding.acquireComponent()

* binding.lookupComponent()

* config.getProperty()

* config.findUtility()

* config.findUtilities()

* config.PropertyMap.getValueFor()

* config.IConfigSource._getConfigData()

* config.PropertySet()

* naming.lookup()

* naming.parseURL()

Also, renamed 'config.LazyLoader' -> 'config.LazyRule' to reduce confusion
with 'storage.LazyLoader', which has a very different purpose/function.

Tutorial edits to sync w/current API.

Stamp out '_provides', replace with 'declareAsProviderOf'.  Removed
extraneous '_provides' declarations left over from the days when classes
could effectively be their own 'binding.New'.  I decided not to document
the interface of attribute binding objects at this time, because the
implementation is tied to the classes right now anyway.

Also, misc. TODO edits.

Finished adding tests for daemons and schedulers.

Added one more "assembly events" test case.  Updated TODO to reflect what
I hope is the final release checklist for 0.5a1.

Moved ZConfig integration from 0.5a1 to 0.5a2; there's just not enough
time for it to make it in this week.  The remaining unit tests, doc fixes
and misc. cleanups are more important for this iteration.

Barring any emergencies, we should reach 0.5a1 within a week.  Updated
TODO to reflect current status; adjusted stream factory signatures so that
'autocommit' is only required to be specified for non-readonly operations.

Misc. TODO updates.  0.5a1 looms nearer!

Began update of TODO to match current 0.5 alpha release plans

First draft implementation of system scheduling proposal.  (Also, misc.
TODO additions.) Interfaces have been streamlined and names changed.
Daemons are now "periodic tasks", and the provided base class implements an
"adaptive task" interface.  "Daemonic app", "event loop", and "daemon
scheduler" are now just a single "system scheduler" interface, which can
schedule periodic tasks as well as one-offs.  I settled on "higher priority
number" = "higher priority" for priority sorting.

I'm beginning to like this whole "asynchronous programming" model that
Twisted provides.  AdaptiveTask (formerly BaseDaemon) is a lot simpler,
MultiDaemon went away, and the system scheduler is only two screens
of code to manage idle timeouts, prioritized task scheduling, and more.
Heck, "shutdown after X" was as simple as 'callLater(duration,self.stop)'!

There is one minor problem in this draft...  if a task raises an unhandled
exception, it won't be scheduled for further re-run.  I'm not sure if this
is a bug or feature yet.  It should be easy to fix, though.

REINSTALL WARNING: a package was removed from setup.py on this change,
so be sure to *delete* your PEAK installation from 'site-packages' before
reinstalling.  Also be sure to update your checkout with 'cvs update -dPA'.

Change notes:

* It's now possible to use 'config.declareModule()' to patch third-party
  (i.e. non-PEAK) modules, and also to pre-declare inheritance or patch
  data for modules inherited via package inheritance.  This was done to
  make PEAK/Zope reusage/patching easier, and also to ease patching of
  large generated-code models (like UML and CWM).

* The patches to add "Pythonic features" to UML models are now in a
  separate module, 'peak.metamodels.core_addons', so that they can be
  shared amongst all UML versions.  This was made possible by the
  new 'declareModule()' function.

* Cleaned up and removed the now-unnecessary subpackage of UML13 that was
  there just to provide a home for the patches that can now be applied by
  "remote control", so to speak.

* 'lazyModule()' now has an extra parameter that can be used to hook the
  initial loading of a module.  This was added so that 'declareModule()'
  can patch modules when they are loaded.

Got rid of deprecated 'peak.model' classes: Model, Package, Reference, etc.

Partial cleanup of '__class_provides__'; now it's supplied by the most
basic metaclass (Activator), instead of being poked in by 'Once' attributes
after the fact.  There probably really should be a 'registerAttribute'
interface method or something for this, but at least now it's not *too*
unbearably ugly.

Added basic plans for 0.5a1 (bugfixes, cleanups, and XMI writing), and
0.5a2 (runtime infrastructure enhancements such as ZConfig support, and
dynamic class mixins).

Removed some completed or no longer applicable "future" items.

Removed misc. completed, outdated, or redundant TODO items.

Added type conversion support for 'ManagedConnection.txnTime'.

Added FacadeDM - a data manager that "fronts" for other DM(s) via a
different key: just override its 'retrieve()' and 'oidFor()' methods.

Support disjoint collections as allowed by the XMI spec.  Misc. TODO
updates to remove stuff recently accomplished.

Added lazy load for persistent object attrs and implemented QueryDM base
class by refactoring EntityDM into QueryDM and EntityDM.  Similarly split
interfaces into IDataManager and IWritableDM.

Added LDAP field type converter support, with unit tests.  This uses the
'peak.ldap.field_converters.*' property namespace, looking up the field
names to find a converter (or import string thereof) that takes two
arguments (name and value) and returns a replacement value.  The value
passed in may be 'None' if no value was present in the retrieved data.
Otherwise, it's a list of values returned from the LDAP directory (except
for the 'dn' field, which is always just a string.)  Converters are
imported from within the peak.storage package, so we can add some
"standard" converters and then reference them with import strings
like '"LDAP.someConverter"'.

AbstractRack -> EntityDM

Misc. typo fixes

Fixed hard tabs in TODO.txt; updated re: Persistence package merge.

Update for current status

Misc. additions/clarifications of TODO list.

SQL error handling and joinTxn() shortcut

* SQL errors are now logged; __traceback_info__ is set, although our
  current traceback formatter for logging doesn't use it yet.  We also
  maybe should log SQL warnings at LOG_WARNING level instead of LOG_ERROR.

* You can now supply joinTxn=(a true value) as a keyword arg to a SQL
  query executed on an SQLConnection, and the connection will join the
  transaction.  This seems cleaner in actual use than a separate .joinTxn()
  call.

* Fixed some issues with Gadlfy not implementing the full DBAPI - or even
  a decent fraction of it.  :(

PropertyName -> peak.api; Improved lazy import facility

* naming.PropertyName is now simply PropertyName; it never really had much
  to do with naming in the first place, doesn't depend on anything else,
  and needs to be accessible in all major subpackages and in end-user API
  calls.  So now it's in-line in the peak.api module.

* Added a 'lazyModule(moduleName)' function to peak.util.imports.  This
  new "next generation" way of creating a lazily-loaded module actually
  creates an object that *will be* the real module in sys.modules, and once
  loaded will be indistinguishable from an ordinary Python module.  (Well,
  under Python 2.3 the type() will be different, but you can't have
  everything.)  I was getting really fed up of tracing into
  lazyImport.__getattr__ during debugging sessions, so I was inspired to
  come up with something that would only have code overhead when it's first
  touched.  Subclassing ModuleType, adding a __getattribute__ method, and
  clever use of the Python 'reload()' function saved the day.

* This now completes all the non-documentation items I want for 0.5a1,
  although I may "work ahead" a little on the features list, especially if
  Ty keeps pressing for SQL type maps and LDAP schemas.  ;)  (In truth, I'd
  like those and some of the other database features Real Soon Now myself.)

pgsql works, and txn integration tested.

Also note that txnTime should be converted when we have SQL result type
converters.

Added [Load on demand] rules section, and made running.clusters use it.
Also added basic unit tests for peak.running.clusters.

Components now support names and paths:

* Objects created by naming system lookup, binding.New(), or
  AutoCreated, can all now automatically receive information
  about their name as well as their parent component.

* binding.getComponentName() gets an object's name or 'None'

* binding.getComponentPath(c, relativeTo=None) gets an object's
  path from its root component, or as a relative path from
  'relativeTo' (assumes 'relativeTo' is a parent of 'c').
  Unknown names are rendered as '*' in the path.

* Added 'binding.Acquire' after all.

* Improved management of _p_jar/_p_oid by model.Element, using
  _p_jar as the parent component, and _p_oid as the component
  name for elements.

* Added unit tests for name and path support.

Added some more to-do items.

Added more details and items to TODO so I won't forget them.  :)

Reorgs and and misc. cleanups; updated TODO for 0.5a1 vs 0.5final

Moved ActiveDescriptor(s) classes from binding.meta to binding.once and the
rest of binding.meta to model.method_exporter.  Moved the method-exporter
tests that lived in metamodels.tests.General to model.tests, where they
really belonged, and added model.tests to the install and API test suite.

Added working support for Gadfly.  Misc. cleanups/fixes and TODO updates.

Reformatted peak.running.clusters docstring to structured text for better
HappyDoc output.  Defined the rule for 'peak.running.cluster._filename' in
peak.ini instead of in the code.

Renamed PropSet to PropertySet, and made it possible to go from a
PropertyName to a PropertySet by calling '.of()' (e.g.
'PropertyName("foo").of(anObj)'.  PropertySets can now reproduce in various
ways, e.g.:

     c = PropertyName('peak.running.cluster').of(None) # -> PropertySet

     c._host['two.baz.com']	# ('even','prime','weird')
     c._groups()		# ('odd','even','prime','weird','qux')

But none of this is documented yet and possibly subject to change.  :)
(Especially since we're probably going to change how the cluster namespace
works anyhow!)

Ideally, it should be easy to go from a property name to its value or to
a property set, and to go from a property set to a value in some namespace.
Of course, it's somewhat questionable whether we'll be doing this for that
many properties in the first place!  And this is all going to go "full
circle" once properties have a naming context that can be used to look
them up in.  I worry a bit about creeping TMTOWTDI-ism in all this.  :(

Reverted the ability to omit schemes/bodies from parsed URLs, since that
might violate the IName interface contract for URLs!  Fixed a problem with
_defaultScheme support.  Made some notes in TODO about issues in the naming
system that need review/investigation.

Made peak.model StructuralFeatures use a uniform API for manipulating
object attributes, which will make it easier to implement "lazy load"
attributes later, and made it easier to implement _p_change support, which
was also added.  IBindingAPI now includes _getBinding, _setBinding,
_delBinding, and _hasBinding methods, and model.Element sets _p_changed
when a modifier method is called.  Also did some more TODO reprioritizing;
it now reflects the 0.5 priority order much better, even though it still
lumps items together by package.

* Added 'binding.Constant(key, value)', which allows you to specify
  a constant value to be used as a property or utility in that
  component subtree.

* Added storage.get/begin/commit/abortTransaction() APIs for ease
  of use in short scripts.  Defined a default provider for
  ITransactionService in peak.ini that provides each component root
  with its own TransactionService instance.  It's easily replaceable
  with other allocation strategies, if it should be needed.  I may
  change this later to a per-root strategy, since the current strategy
  requires creating localConfigs for each component root.  We'll see
  what actual use has to say about the best way of doing it.

* Fixed a problem with local/global config object startup that could
  cause "phantom" configuration objects to appear via recursion of
  getLocal() or getGlobal().

* Got rid of binding.meta.assertInterfaces, which is no longer needed
  because the Interface package now correctly handles metaclass
  instances.

* Got rid of Singleton, since it's not used for anything and I could
  neither think of any use cases nor remember why I added it in the
  first place!

* Made it possible for CachingProviders to cache per-localConfig as well
  as per-object.  This is handy for implementing per-hierarchy services,
  like the TransactionService provider mentioned above.

* Misc. TODO updates.

Made Model.Element Persistent.  There is still an unresolved issue relating
to mixing Persistent with other metaclasses under Python 2.2, however.  Be
wary of mixing Persistent with *any* PEAK class until it's fixed.  :(

Added error logging to basic txn error handler.  Fixed a couple of bugs
exposed by the newly added error logging.  :)  Reorganized TODO to better
reflect priority of remaining items, although it's not completely
prioritized yet.

Re-org/refactoring of many items from the TODO list for 0.5, including:

* moved peak.binding.imports -> peak.util.imports, added importSuite()

* Decided not to do binding.Acquire, it should just be a documented idiom

* Moved LDAP and lockfile URLs to peak.storage.LDAP and
  peak.running.lockfiles, respectively

* Moved factories.getObjectInstance -> spi, updated peak.ini to load
  naming.spi as the provider.

* Consolidated naming.references into naming.names

* Made ParsedURLs compile their pattern strings, and automatically make
  their 'fromX()' methods classmethods (and the same is true for other
  'struct' types.

* Refactored cursors and connections out into distinct SQLCursor/LDAPCursor
  types, based on a common AbstractCursor.

* Added TooManyResults/TooFewResults errors to peak.exceptions

Completed configuration files refactoring.  Configuration files now can
have [Load Settings From] and [Provide Utilities] sections, and property
sections can now define rules instead of just values.  Property values are
eval()'d at rule access time rather than configuration loading time, and
they have access to 'propertyMap', 'propertyName', and 'targetObj' locals,
as well as the full peak.api.* and config_components' globals.  "Load
Settings From" sections can access a potentially infinite variety of
sources for loading configuration data; the sources are defined using
the 'peak.config.loaders.*' property namespace, and thus can be
meta-configured even in the same configuration file.  Builtin loader types
are 'file' and 'mapping'.  GlobalConfig no longer has any hard-wired
configuration steps; everything is now in the peak.ini file.  Subclasses
of GlobalConfig need only perform 'self.config_filenames.append(fname)'
before calling GlobalConfig's 'setup()' method, to cause additional config
files to be loaded after 'peak.ini'.

Everything that is configurable in PEAK, is now (in principle) configurable
via property files.

Added more docs and a test to the FileParsing framework.  Refactored
AbstractConfigParser to make it easier to define how differently-processed
configuration sections will be handled.  Misc. TODO additions.

Added detailed plan of attack for 0.5 release.  It's getting closer...

A first pass at cleaning up the TODO file.  Also, removed the Specialist
class and associated interfaces altogether, since Racks can serve all the
same functions in PEAK.  (Perhaps Rack should be renamed to Specialist, but
that's a matter for future debate.)

'setupModule()' and 'adviseModule()' will now issue warnings for most
questionable code structures and variable redefinitions that might not
work the way you'd expect or intend under module inheritance.

Added tests and documentation for 'adviseModule()' API

Added warnings for detectable module-level modifications of mutables
in modules which are used for inheritances or advice.  Added an API
function, 'configure(object, attr1=val, attr2=val,...)' to safely
set attributes of mutables that might have been defined in a derived
module.  Also, misc. updates to TODO.

Improved key integrity checks in DataModel: Record objects now disallow
modification of key fields unless the old value is None, and cache
collisions between records with supposedly unique keys will result in an
AssertionError.  Also, made the requirement of an LDAPConnection object
more visible in Database.LDAPModel, and made misc. CHANGES/TODO updates.

Fixed the "reference to rebound class within another class" problem with
module inheritance, as reported by (who else?) Ulrich Eck.  :)

Added some notes on ideas for what might go into version 0.3.

Moved TODO.txt out of the docs directory so HappyDoc will make an HTML file
for it in the reference docs.