https://wiki.openstack.org/wiki/Grenade

Grenade is a test harness to exercise the OpenStack upgrade process between releases. It uses DevStack to perform the initial OpenStack install and then walks through upgrading each enabled project.

Source

IRC is done in #openstack-qa

A complex of bash scripts that manipulate DevStack installation to create a base installation, run Tempest and then piecemeal upgrade the various components and run Tempest again.

A notional blueprint has been started but there is a new blueprint process coming together where blueprints will be based on this template and then reviewed via the ReviewProcess.

Some things I'm not sure about or not seeing yet:

• ceilometer is not supported in DevStack by default and it seems that Grenade is not fully ceilometer aware, but could be
• It's not clear how before and after tests are managed, if at all. The goal with ceilometer tests would be to take some samples of some metric in the base installation and then compare those with samples of the same metric in the upgrade installation to see if they make sense.

This suggests a bit of a plan of action:

1. get ceilometer in there in the first place, never mind the details of the comparison
2. then see about the comparison

Doing 1 might just as easily be making ceilometer default in DevStack?

There doesn't appear to much ceilometer testing in tempest. Is this true? Yes, this is because of performance problems with sql-a in ceilometer. Currently the testing requires sql-a. There are a pile of unlanded tests.

I'm now the driver for the #5 gap.