Other nose/attr

Table Of Contents

Previous topic

Nosetests Introduction

Next topic

Running Tests Using dybinst

This Page

Daya Bay Links

Content Skeleton

Using Test Attributes

As test runs get longer it becomes very useful to control which tests get run in a flexible manner. This functionality is based on the nose attrib plugin documented at nose.plugins.attrib

Package Level Nosetesting with attributes

Example based on simple and quick to run dybgaudi:Database/DybDbi/tests/test_feecablemap.py for easy checking.

from cnf import setup, teardown
from DybDbi import GFeeCableMap


def test_spin():
    r = GFeeCableMap.Rpt()
    print len(r)
    print r[0]
    for i,o in enumerate(r):
        print " %3d feechannelid %d feechanneldesc %s feehardwareid %d sensorid %d sensordesc %s pmthardwareid %d pmthrdwdesc %s " % \
              ( i, o.feechannelid.fullPackedData() , o.feechanneldesc, o.feehardwareid.id() , o.sensorid.fullPackedData(), o.sensordesc, o.pmthardwareid.id(), o.pmthrdwdesc )

def test_spin_slowfake():
    r = GFeeCableMap.Rpt()
    print len(r)
    print r[0]
    for i,o in enumerate(r):
        print " %3d feechannelid %d feechanneldesc %s feehardwareid %d sensorid %d sensordesc %s pmthardwareid %d pmthrdwdesc %s " % \
              ( i, o.feechannelid.fullPackedData() , o.feechanneldesc, o.feehardwareid.id() , o.sensorid.fullPackedData(), o.sensordesc, o.pmthardwareid.id(), o.pmthrdwdesc )
test_spin_slowfake.minutes = 10


if __name__ == '__main__':
    setup()
    test_spin()
    test_spin_slowfake()
    teardown()

The below line assigns a minutes attribute to test_spin_slowfake.

test_spin_slowfake.minutes = 10

Subsequently can select which tests using an attribute expression:

nosetests -v -A "minutes > 5"
nosetests -v -A "minutes < 5"

Real command examples from DybDbi package directory:

  • DBCONF=offline_db nosetests -v -A "minutes > 5"

    run only tests for which the attribute expression is true : currently only 1 test

  • DBCONF=offline_db nosetests -v -A "minutes < 5"

    run only tests for which the attribute expression is true : currently 270 tests

  • DBCONF=offline_db nosetests -v

    run all tests in the package : currently 271 tests

  • DBCONF=offline_db NOSE_EVAL_ATTR="minutes > 5" nosetests -v

    using environment controlled attribute setting : runs 1 test

  • DBCONF=offline_db nosetests -v tests/test_feecablemap.py -A "minutes > 5"

    runs just test_spin_slowfake

  • DBCONF=offline_db nosetests -v tests/test_feecablemap.py -A "minutes < 5"

    runs just test_spin

Testing at dybinst level

Analogously to the above (from dybsvn:r11731 ) the control can be done at dybinst trunk tests <pkg-or-alias> level with:

  • ./dybinst trunk tests dybdbi

    run all tests in dybdbi package, currently 271

  • NOSE_EVAL_ATTR="minutes > 5" ./dybinst trunk tests dybdbi

    only tests meeting the expression, currently 1

  • NOSE_EVAL_ATTR="minutes < 5" ./dybinst trunk tests dybdbi

    only tests meeting the expression, currently 270

Testing at bitten slave level

Commit Message controlled deep testing

Attribute expressions attr:<expr> in svn commit messages like the below are detected and passed to nosetests:

example commit message that triggers long tests only attr:"minutes > 10"
example commit message that triggers medium tests attr:"5 < minutes < 10"

Warning

a build must be triggered within 60 min of the commit time for the attr: command to take effect

Protecting quotes is required, eg with:

svn ci -m '  attr:"minutes > 5" '

Example exercising the machinery, which demonstrates how the bitten-slave access the SVNLOG_attr parsed from commit messages.

./dybinst -E demo.sh trunk tests dybdbi     # demo.sh contains export statements

Periodic deep testing based on revision number

BUILD_REVISION is available to dybinst from dybsvn:r11732 and is used to set default attribute expressions that select nosetests from dybsvn:r11738.

build revision default expression
ends with 00 minutes < 101
ends with 0 minutes < 11
otherwise minutes < 6

Examples of how to exercise the machinery:

BUILD_REVISION=12345 ./dybinst trunk tests dybdbi
BUILD_REVISION=12300 ./dybinst trunk tests dybdbi
BUILD_REVISION=12340 ./dybinst trunk tests dybdbi

Note

commit message attr: commands trump BUILD_REVISION defaults