Good-Py: Known Good Versions Manager for Python and zc.buildout

Good-Py is a web application hosted on Google App Engine that aims to help manage known-good version sets of Python packages.

Why do we need this?

In the Zope and Plone worlds, we depend on large set of eggs (in the setuptools sense) published on PyPI (pypi.python.org) and other indices. Because setuptools (and zc.buildout) will typically prefer the latest release for a given package, it is easy to end up in a situation where an application ends up pulling in a too-recent release that does not work well with the other packages in the system.

This problem can be solved with pinning all releases to a "known good set". With zc.buildout, this is typically achieved using a [versions] block in buildout.cfg like:

[versions]
zope.interface = 3.4.0
zope.component = 3.4.0

Such a versions block can be set in a separate file that is included from the buildout:

[buildout]
...
extends =
    http://dist.plone.org/release/3.3rc2/versions.cfg
    http://www.example.org/release/some-system/1.0/versions.cfg
versions = versions

[versions]
some.package = 1.2

This example shows some important aspects of the version pinning system:

In the example above, some-system is a fictional plugin to Plone that contains additional dependencies, including some newer versions than what are included in Plone.

What does Good-Py do?

The Good-Py application aims to simplify the management of known-good sets, by allowing a [versions] block to be calculated automatically based on various predicates. If 'some-system' were using Good-Py, the 'extends' block could look like this:

[buildout]
...
extends =
    http://dist.plone.org/release/3.3rc2/versions.cfg
    http://good-py.appspot.com/release/some-system:1.0?plone=3.0&archetypes=1.5.1
versions = versions

The web application at http://good-py.appspot.com/release calculates the versions block for the given version of the system (1.0). It is also possible to specify a target platform (Plone 3), to indicate that you need a version of the dependencies that will work with the given platform.

The versions block is calculated from a number of predicates of the form:

known-good: foo.bar = 1.2.3
This package is a known good version for the given version of the system. An unlimited number of such predicates may be stored against each system version. The most recent package version will be included.
known-bad: foo.bar <= 1.2
A version specifier for packages that definitely will not work. Again, any number of such predicates may be used. The various setuptools version qualifiers are allowed: e.g. <=, >=, <, >, and ==, and multiple versions can be combined using commas, e.g. foo.bar <= 1.2,>3.2.
extends: other-system = 1.0
Imports all predicates from another system at a given version.

The generated [versions] block will include:

Note that platforms specified on the query string are only considered as constraints. That is, the version set returned will not violate any constraints from the platform any system/version specified, but the known-good predicates from the platforms are not included either. You can always load the platform version explicitly with another 'extends' line, however.

How is it managed?

The Good-Py application includes an administrative interface with the following features:

Security is managed with Google Accounts. A list of accounts with edit rights is maintained against each system. The application administrator can change any system or version, and may also flush the cache globally.

Who's responsible?

This application was created by Martin Aspeli (optilude at gmail dot com)