ploneout: workingenv and buildout working harmoniously

Having been busy with the opencore redesign for a while now, it's been awhile since I worked on plone3 or looked at the build situation with plone. When I sat down to make some simple changes to wicked, I wasn't surprised that everything in my old plone3 dev environment broke with an svn up.

I *was* pleasantly surprised to discover that I could install buildout and ploneout into a workingenv and get a running zope/plone3 instance without any problems.

Kudos to those responsible whoever you may be!

wicked into plone

Finally! A little late but it's in and waiting for framework team and wiggy's approval. Was pretty happy at how little I actually had to touch plone or any of it's dependencies to get this done.

http://dev.plone.org/plone/changeset/12050/ - http://dev.plone.org/plone/changeset/12057/

check it out, let me know if you find anything.

see: http://svn.plone.org/svn/plone/bundles/3.0/ for instructions on setting up plone3(an automated build is coming via hanno)

wicked itself is available from the cheeseshop ( http://cheeseshop.python.org/pypi/wicked/1.1 ) or svn ( http://svn.plone.org/svn/collective/wicked/trunk )

There are some cool things under the hood in this release: directives for specifying exactly what field should be wicked on any existing content type, examples of multi-content type application of the filter, media wiki style bracket linking(optional), and fieldevent, a simple system for extending the render and storage of any field object's value. Fieldevents could be particularily useful to anyone interested in any sort of wicked like behavior such as xslt pipelining, inline rendering of referenced content, pagination, etc. They could be used to add css id's and classes to existing markup that KSS could then bring alive.

Looking forward: the last little bit is to fix up the old Product style wicked package to handle old instances.

Though 1.1 was developed against zope 2.10 (the plone integration depends on the persistent component registries), I suspect it will work just fine with zope 2.9. Products.wicked will essentially become a registry for calling in the necessary zcml from wicked.at and wicked.atcontent.

OpenPlans is planning some simple syntax extensions soon and soon we will need to integrate wicked into some non-zope apps (particularily excited about this). In 1.1, I stubbed out a centralized cache / link map that would effectively let wicked get away from at references and the first step to moving it toward being more of a service rather than a zope or plone bound extension.

PSA for PTC users

Important news for everyone using PloneTestCase trunk and Five::

Debugging PTC Trunk

Some minor things have changed on PloneTestCase trunk that will create breakage for some common but questionable test setups. This outlines the breakage and how to fix it.

Suffice it to say this: death to installProduct('Five').

#!bootstrap/environment

summary: Eggs are changing how things are done around these parts. This entry will cover how to use Ian Bicking's workenv with Zope2 and take a sideway glance at zc.buildout.

Hello Eggs

One of the first things plone developers noticed about eggs was the funny way they (by default) wriggle into your python's site-packages. Next thing noticed: the zope testrunner didn't know where to find them without symlinking them into Zope's lib/python.

Zope(2) classically did not handle it's mountains of python like the rest of the python world did, so real surprise that zope did not jive with the new setuptools regime. But things are getting better...

Building an Environment

Jim Fulton, pope of zope, has been working on a system for composing large complex systems called zc.buildout . As part of this recipe driven strategy it can install eggs into a local buildout directory(along with the rest of the buildout "parts"). Ostensibly, we will probably see recipes that can construct a particular type of development environment for any variety of plonehead in the not too distant future.

zc.buildout is a bit industrial to the first blush. Nothing wrong with that, just a bit of framework comprehension and possibly simple programming required(if the current menu of recipes doesn't fit your bill). zc.buildout looks very powerful and simple for deployment tasks, including building systems beyond python like squid or Apache. But maybe not the tool for playing around, testing and discarding your python ideas (yet).

note: If you are interested in zc.buildout for zope2, take a look at topp.buildout, an initial experimental buildout attempt for opencore stack. For pedagologicaly purposes only. Ian says this is a bad buildout example. look at zc.sharing.

Ian's workingenv is a bit to the other side of the same spectrum. It does a bit of the same thing but is more focussed on the python developer and python packages (as opposed to say a zope deployista with a host of other applications). It is more concerned with the problem of keeping those snakes on the plane than the care and feeding of your squid.

workingenv allows you to create a temporarily constrained environment for python, substituting a localized work area for your site-packages. It plays nice with setuptools and is generally helpful.

the self contained script runs like so::

$ workingenv.py mysandbox

This bootstraps a local installation of setuptools to 'mysandbox', which now has it's own bin, lib/python, conf and src directories.

To activate the environment(using bash)::

$ source mysandbox/bin/activate

Once activated, you can install eggs like so (mysandbox/bin/easy_install is now part of your path)::

$ easy_install zc.buildout

All scripts are installed to the local bin. Likewhise, any source packages you download and install via 'python setup.py develop' are installed in the local python directory. For example::

$ cd mysandbox/src
$ svn co https://svn.openplans.org/svn/tagger/trunk tagger
$ cd tagger
$ python setup.py develop

This puts an entry in mysandbox/lib/python/easy_install.pth pointing to my development directory in src/tagger.

Even more convenient, workingenv.py will take a -r argument for an egg reciper. This recipe may either be a url or a file. To bootstrap a local turbogears development environment::

$ workingenv.py mytgsandbox -r http://svn.colorstudy.com/home/ianb/workingenv/tg-0.9.txt

Pretty nifty, eh? You may also embed -r into the recipes allowing recipes to be aggregates.

workingenv and Zope 2

For working with zope2, we can use workingenv's --env flag to set the INSTANCE_HOME and SOFTWARE_HOME environmental variables(needed for running tests without zopectl). Zope install also expects the disutils --home flag, so we add it too.

In this example, I create the environment first, pointing it at a zope src directory I have not created yet, then checking out the zope and installing zope that directory.

$ workingenv.py myzopesandbox --home --env=INSTANCE_HOME:~/mysandbox/zope --env=SOFTWARE_HOME:~/mysandbox/zope/lib/python
$ cd myzopesandbox; source bin/activate
$ svn co svn://svn.zope.org/repos/main/Zope/branches/2.9 zope
$ cd zope
$ make inplace; make instance

This almost works. For zopectl to see the lib/python inside the workingenv sandbox, you have to edit line 8 to read::

PYTHONPATH="$SOFTWARE_HOME:$PYTHONPATH"

Now you have a fully functional constrained python environment for zope. To deactivate::

$ deactivate

Python and shell return to their normal search path.

The Zope Situation

Not too shabby but not a great situation with zope. Obviously the match is a bit rough right now. Editing a script is easy to forget, and all of Zope's legacy codebase exists as products, not eggs.

This particular place where the road turns to dirt is where I think this may zc.buildout and workingenv might be able to work together for best effect. zc.buildout lets us write recipes for zope2 installation and for handling products. The recipes could recognize when they are executed in a workenv setting(workingenv.py sets an environmental variable) and smooth such rough spots as adding the current PYTHONPATH to zopectl's PYTHONPATH.

TestRunner

And maybe recipes could set the testrunner up to find the right eggs. Currently, I have only found one sure fire way to have my testrunner run tests for a "development" egg (an egg installed via python setup.py develop)::

For the tagger install example above(if it were installed into the zope sandbox)::

$ cd myzopesandbox
$ zope/bin/zopectl
zopectl> test -s tagger --test-path=/myhome/myzopesandbox/src/tagger/src/tagger

Part of the issue is the zope testrunner isn't really egg/setuptools aware yet(so it doesn't see those nifty pth files and follow them). The other part of the issue, is that having the testrunner search the entire site-packages is slow. This is the part workingenv solves; it's ok to search all available python pkgs because the environment is constrained to what you are working on.

Final Thoughts

I'd like to see the testrunner handle setuptools better and I see potential in using zc.buildout to handle the ugly bits of bootstrapping one's environment. I'd like to see both of this tools take advantage of workenv and it's developer and setuptools friendly way of doing it's job.

I look forward to more recipes for zc.buildout and a decent tutorial(nouri? bitte?). Unlike workingenv, it's harder to get zc.buildout to doing something useful from the get go in the raw dev environment.

My response to workingenv and zc.buildout is mostly viceral though; in the land of thousand frameworks, a "script that just works " like a cool glass of water. We have deployment needs at work, but more commonly we have the need for bootstrapping consistent development environments. workingenv plays nicely with existing tools like easy_install amd setup.py whereas zc.buildout requires everything to go through the buildout script.

Right now, I prefer workingenv for developing and we are using zc.buildout for our buildbot. Still would like convergence though. Seems possible too.

More on Layers

Riding Off into the Layered Sunset

ZopeTestCase still needs to grow up. Depending on code execution by import is pretty evil and forces developers to make choices that severely limit the ability of a framework to adapt to change(like say....adding zope3 component architecture to the mix). Controlling the exact order of import when dealing with code outside of your control is near impossible. The longer this remains as such, the more the engineering debt compounds and the worse someone will get hurt.

In other words, do you want me to have to break your tests again and again? Probably not.

Does ZopeTestCase have to die? Maybe not. Maybe just get it's act in line with the current state of affairs.

Following PhiliKON's suggestion that ZopeTestCase should have it's own layer that handled installProduct calls, I did some preliminary investigation into what it would take to put the entire plone stack's testing onto layers including all of ZopeTestCase. The applied initial fruits of that investigation is the stop gap measure described in the last post.

The complete work in progress is available at the following locations and goes much deeper into using layers to simplify the test structure:

• Zope 2.9 (layer enabled ZTC and Five)
  
  
• Zope 2.10 (layer enable ZTC and Five)
  
  
• Plone 2.1 for z2.9 (w/ all dependencies layer enabled and layer enable Five)
  

In these brancheZopeTestCase has been refactored to isolate all behavior formerly triggered by import. In most cases, calls that were intended to act immediately defer there action until the appropriate layer via simple module level registries. This pattern is carried to PloneTestCase and other ZTC derivatives; for example, PloneTestCase has it's own layer that handles setting up the plone site the tests run against.

The PTC layer inherits from the ZTC one and therefore all plone tests run as a section of the zope test case layer.

The majority of the tests run without issue and most issues can be fixed with by removing code. More people need to bang on it, but it's ready to be banged on.

Please bang away and let me know(or continue the work. would love some help here). With some feedback, this could get into Zope 2.11(only 3 months off).

Finally, a big shout out to Stefan Holek. Without ZopeTestCase(and stefan to impress upon people the proper way to use it), many of us might still be rubbing rocks together in a dark cave, sacrificing hours to the zope gods hitting reload.

Like many things in this world of code (like Zope2, AT, CMFFormController, etc), if it didn't work so well for so long we wouldn't be here to bitch about it now.

-w

I Break Tests Because I Care

First, a sincere apology to all of you who woke up this morning, updated your Plone bundle, ran your tests and suffered unexplainable breakage. Thank you to those of you(particularily the observant stalwart developers of Zest Software) who provided feedback and helped identify the following Zope 2 bug.

It's 4am, do you know who your tests are bad touching?

Lately I've been working on the testing frameworks that are derivative of Testing.ZopeTestCase. You may have read that ZopeTestCase must die. You may have received a tongue lashing from a Five developer about writing a test that uses the infamous "installProduct('Five')". You may simply have used the provided test framework unaware that your tests were treating other packages' tests like the Bush Administration treats personal civil liberties or poor people and minorities.

It's ok.

If you can forgive me for breaking your tests, I'm sure all those hard working Five developers will find it in their hearts to forgive you for the fact you can't run Plone's test and Five's tests at the same time.

Until now (or about six months ago shrouded in the vault of zope.*), the right way was hard and painful and full of stinky boilerplate.

Layers: The Neighbourly way to Set Up Your Zope 3 Components

In zope 2.9, the zope 3 testrunner became available to us z2 trogdolytes and with it the zope3 strategy for more efficient functional and integration testing:layers.

Layers are a little like social clubs for tests. Just because your unit tests in your package live next to your integration tests doesn't mean that they should run together or even like each other. Layers let everyone run where they are comfortable and happy and passing; the frat layers spills no beer on the goth layer, the goth layer doesn't bum out the church goers and so on. This applies whether you run your tests alone, or with everyone elses.

The testrunner groups test cases (think classes like ZopeTestCase, PloneTestCase etc) by their layer and runs them together (any tests lacking a testcase with a layer run in a group called unit tests). Between the setup and the teardown of a layer, all tests run inbetween are assure of having the environment they expect.

Cut to our dilemma. What we are currently most concerned about in this picture is the component architecture. The entire component architecture: all the views, adapters, events, etc that make up a running Zope 2 site. If you have ever had to debug a rogue
ComponentLookUpError,

spent hours wondering where your views and adapters disappeared to, you know this pain.

The current method of doing this, Testing.ZopeTestCase.installProduct('Five'), can only be called once and only once to set up the component architecture; all subsequent calls have no effect once Five has been loaded by ZopeTestCase. Therefore, If a unit test happens to call zope.testing.placelesssetup.tearDown or zope.testing.cleanup.cleanUp, all following tests will have no component architecture. Welcome to test debug hell.

Fear not. There is hope for your fragile testing setup. layers can setUp and tearDown the CA as often as needed.

PloneTestCase and CMFTestCase now have some basic layer support to make this easy.

• http://svn.plone.org/svn/collective/CMFTestCase/trunk/
  
  
• http://svn.plone.org/svn/collective/PloneTestCase/trunk/
  
  

Any test that inherit from these classes gets the layer treatment that handle setting up the CA. Usage is usually as easy as simply putting these branches in your products directory.

Since layers are a useful tool to the test writer, let's take a looK
at what is going on here. It's best to simply think of the layer as a named data block with 2 methods: setUp, and tearDown.


Here we see the layer that loads the CA. It is an old style class with a classmethod setUp loads the entire CA(and does a bit to make sure ZCML error messages are not supressed due to ZTC optimization). The classmethod tearDown cleans up afterwards.


Pretty simple.

The classmethod calls may look a bit strange. Again, think of this class as data block rather than an object factory; the layer will never be instanciated in the classic OOP sense.

Note: The conventional pattern for layer is to use an old style class and classmethods; concievably, you could use a module with setUp and tearDown functions or an instanciated object with setUp and tearDown methods, but an old style class with classmethod seems to be the most reliable and flexible.

Applying the layer is easy too. The layer is assigned as an attribute to the test class. Currently for BBB a set of checks makes sure that layers are available(hence setup.USELAYER). If you know you are developing against 2.9 or greater and don't need any BBB, this check is unnecessary.

Now when we run the the Plone tests ($ bin/zopectl test -s Products.CMFPlone), we see the following ::

Running unit tests:
Running:

......................................................................................................................

Ran 158 tests with 0 failures and 0 errors in 1.465 seconds.

Running Products.PloneTestCase.layer.ZCMLLayer tests:

Set up Products.PloneTestCase.layer.ZCMLLayer in 0.883 seconds.

Running:
..................................................................................................... etc....

Ran 1750 tests with 11 failures and 0 errors in 182.200 seconds.
Tearing down left over layers:
Tear down Products.PloneTestCase.layer.ZCMLLayer in 0.002 seconds.
Total: 1908 tests, 11 failures, 0 errors

All tests not requiring the CA or tests which handle all setup and teardown inside the test run first. Then ZMCLayer is set up, all tests that use it run, and the layer is torn down. If you just wanted to run the CA dependent tests you could do so with the following command:

~/zopesite/ $bin/zopectl test -s Products.CMFPlone --layer=Products.CMFPlone.layer.ZCMLLayer

What does this mean to me and my work?

if you are lucky, it may just be a few more statistics spit out of your test runner (and the ability to run all your tests at once). It could just work in many situations.

Chance are it won't work if your [Plone/CMF]TestCase test commits the following crimes against nature (check yourself if you've been wading in the z3 soup).

If your tests uses PloneTestCase and does the following:

1. uses Products.Five.zcml.load_string or Products.Five.zcml.load_config
   
   
2. uses aforementioned to load functions to load zcml that never gets loaded on Zope site loading
   


3. calls provideAdapter or other convenience registration functions
   


4. calls zope.testing.placelesssetup.tearDown or zope.testing.cleanup.cleanUp
   

1-3 will result in erratic behavior for tests within the ZCMLLayer layer. #4 will result in ComponentLookUpErrors due to the clearing of the registries of the component architecture.

Never fear: just following these easy refactorings.

You must(fixes in order of issues):

1. Delete it. you don't need it anymore
   
   
2. Slap yourself for doing this. Refactor your test into units and integration tests.
   
   
3. Repeat fix for #2
   
   
4. Delete it. you don't need it.
   
   

If you find that you are slapping yourself alot, you are feeling layers are in forcing us to use better testing practices.

You may even find you need a special layer of specific zcml loads. Mostly, you should just be deleting now useless boilerplate and loving it. If not, let me know.

You may notice that layers force us to divide our tests into more appropriate groupings of unit tests(atoms of code), and groups of p[roper integration tests(framework w/ code units).

It is important to understand that with this strategy all the active zcml is loaded for a site. This means if some product or package overrides zcml effecting your test, your test may behave differently than you expect(hopefully, your test coverage will actually act like a canary in the coal mine in this situation).

Note:for working ZTC doctest layer support, a fresh checkout of the Zope 2.9 branch is required.

Coming soon: The Future!!!!

zX

maybe zope should take a hint from OS X and use this fine release to dump some name baggage? 2.10 looks awesome.

just a few hings I'm excited about:

* wsgi & twisted

* formlib

* viewlets

* clockserver

* testbrowser

* z3 ZPTs

* no more traversable directives!

* pythonproducts: zope2 code that lives on the pythonpath like normal python

* native egg support

Zope.org - Changes.txt