Other sop/dbtest

Table Of Contents

Previous topic

DB Validation

Next topic

DB Administration

This Page

Daya Bay Links

Content Skeleton

DB Testing

This section covers the testing of the meaning of DB update entries. Generic machinery validations are described in DB Validation

Numerous very time/CPU expensive problems have occured with the meanings of DB update entries. Furthermore exactly the same problems have occured multiple times and in several cases trivial errors have managed to get into offline_db that have required subsequent re-processing to be abandoned.

In the light of these time wasting mistakes, strict enforcement of the testing of DB updates has been deemed to be necessary.

Nosetesting reminder

As an introduction to nosetesting see Nosetests Introduction which lists references and examples.

When and where

The appropriate time and place for checking the meaning/correctness of DB updates is within the tmp_<username>_offline_db of the updaters immediately after updates are performed there.

The appropriate way to perform such checks is via nosetests that can be directed at the the desired DB via the DBCONF mechanism, typically (for bash shell):

DBCONF=tmp_<username>_offline_db nosetests -v
DBCONF=offline_db                nosetests -v

OR for csh(tcsh):

setenv DBCONF tmp_<username>_offline_db
nosetests -v

More details on DBCONF can be found at N ways to set an envvar. For how testing fits in with the SOP workflow, see Workflow Outline.

Standardized testing

The nosetests corresponding to DB table updating follow the same standard layout as all NuWa nosetests. They are required to be maintained in the tests directory of the package that contains the scripts that perform the update and it should be a sibling to the “cmt” directory of the package. For example:

Automation of testing relies on adherence to standards for test naming and layout.

Responsibility for maintaining tests

The roster of responsibility for maintaining these DB updating tests are, in decreasing order of responsibility:

  1. DB updaters + authors of DB updating scripts, who know best how the results were derived and likely problems
  2. Direct downstream users, typically authors of services that use the results who are well placed to know the constraints that should be applied.
  3. Working group conveners, responsible to steer the above workers and set expectations for testing
  4. Database/testing experts, who know best how testing can be efficiently structured and can advise on techniques to improve test coverage.
  5. Anybody else who finds a problem with results, should add tests encapsulating the finding

Responsibility for running tests

Although multiple persons are involved with maintaining the tests of DB updates the responsibility to run the tests and demonstrate that the tests were run remains with the DB updaters.

Proof of testing

In order to prove that testing has been done, run commands such as the below and copy and paste the text output to be included into your email requesting propagation to offline_db:

date ; pwd ; svnversion .


Arrange a clean SVN revision, by committing any changes and updating working copy

For bash shell:

DBCONF=tmp_offline_db nosetests -v
DBCONF=tmp_<username>_offline_db nosetests -v      ## depends on ~/.my.cnf section names

For csh(tcsh) shell:

setenv DBCONF tmp_offline_db
setenv DBCONF tmp_<username>_offline_db    ## depends on ~/.my.cnf section names
nosetests -v

If deemed appropriate the coordinates of your tmp_ DB can be shared to allow other stakeholders to run the tests.

Once the tests run without error, you can proceed to making dybaux commits The text output proving successful test runs must be included in your mail to Liang requesting the propagation of dybaux commits into offline_db.

Dealing with mistakes

Even with these procedures problems will inevitably continue to get through. When they do the requirement will be to add nosetests that capture the issue. This should avoid past issues coming back to haunt us, as experienced in dybsvn:ticket:1282

Checking entries make sense

Constraining entries to meet expectations of normality will vary greatly by table. However some simple starting points could include constraints on

  1. number of distinct values of identity entries
  2. mean/min/max values of parameters
  3. values of quantities derived from fits to the parameters
  4. differences in parameters between updates expectations on allowable mean/min/max and deltas

Status of packages with tests

Packages containing tests and commentry on the nature of the tests, remember that tests should be sensitive to external DBCONF envvar.

Comparing python datetimes with DBI TimeStamps

For some tables such as CalibPmtFineGain the CWG has a policy of requiring validity TIMEEND to normally be TimeStamp.GetEOT() a standard far in the future date, corresponding to INTMAX = (1<<31)-1. Often tests use DybPython.DB which returns python datetimes. The below approach sidesteps timezone complications by converting the datetimes into TimeStamp with TimeStamp.fromAssumedUTCDatetime:

[blyth@belle7 ~]$ DBCONF=tmp_offline_db ipython
Python 2.7 (r27:82500, Feb 16 2011, 11:40:18)
Type "copyright", "credits" or "license" for more information.

IPython 0.9.1 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object'. ?object also works, ?? prints more.

In [1]: from DybPython import DB

In [2]: db = DB()

In [3]: rec = db("select * from CalibPmtFineGainVld order by SEQNO desc limit 1 ")[0]

In [4]: rec['TIMEEND']
Out[4]: datetime.datetime(2038, 1, 19, 3, 14, 7)

In [5]: from DybDbi import TimeStamp

In [6]: TimeStamp.fromAssumedUTCDatetime( rec['TIMEEND'] )
Out[6]: Tue, 19 Jan 2038 03:14:07 +0000 (GMT) +        0 nsec

In [7]: TimeStamp.fromAssumedUTCDatetime( rec['TIMEEND'] ).GetSeconds()
Out[7]: 2147483647.0

In [8]: TimeStamp.GetEOT()
Out[8]: Tue, 19 Jan 2038 03:14:07 +0000 (GMT) +        0 nsec

In [9]: TimeStamp.GetEOT().GetSeconds()
Out[9]: 2147483647.0

In [10]: TimeStamp.GetEOT().GetSeconds() == TimeStamp.fromAssumedUTCDatetime( rec['TIMEEND'] ).GetSeconds()
Out[10]: True