Other aop/dbsvn

Table Of Contents

Previous topic

Offline DB Backup

Next topic

Bitten Debugging

This Page

Daya Bay Links

Content Skeleton

DBSVN : dybaux SVN pre-commit hook

DBSVN is a script used by the dybaux SVN pre-commit hook to perform basic validation of database updates. This section describes how to test changes to the dybgaudi:DybPython/python/DybPython/dbsvn.py script.

Example of failure

Running dbsvn.py with no change yields error, as no update is not a valid update:

[blyth@belle7 ~]$ rm -rf tmp_offline_db ; svn co http://dayabay.ihep.ac.cn/svn/dybaux/catalog/tmp_offline_db   # pristine lastest dybaux tmp_offline_db
[blyth@belle7 ~]$ dbsvn.py ~/tmp_offline_db -M
Traceback (most recent call last):
  File "/data1/env/local/dyb/NuWa-trunk/dybgaudi/InstallArea/scripts/dbsvn.py", line 4, in <module>
    main()
  File "/data1/env/local/dyb/NuWa-trunk/dybgaudi/InstallArea/python/DybPython/dbsvn.py", line 586, in main
    dbiv()
  File "/data1/env/local/dyb/NuWa-trunk/dybgaudi/InstallArea/python/DybPython/dbsvn.py", line 478, in __call__
    self.validate_update()
  File "/data1/env/local/dyb/NuWa-trunk/dybgaudi/InstallArea/python/DybPython/dbsvn.py", line 400, in validate_update
    assert 'LOCALSEQNO' in tabs, "No LOCALSEQNO in %s " % tabs
AssertionError: No LOCALSEQNO in []
[blyth@belle7 ~]$

Fabricate pre-commit working copy

Last five commits:

[blyth@belle7 ~]$ svn log --limit 5 ~/tmp_offline_db
------------------------------------------------------------------------
r5433 | zhanl | 2013-03-21 10:42:45 +0800 (Thu, 21 Mar 2013) | 1 line

fastforward updates following offline_db rloadcat of r5432 OVERRIDE
------------------------------------------------------------------------
r5432 | zhanl | 2013-03-21 10:41:00 +0800 (Thu, 21 Mar 2013) | 1 line

OVERRIDE fill blind flux
------------------------------------------------------------------------
r5431 | zhanl | 2013-03-21 10:37:57 +0800 (Thu, 21 Mar 2013) | 1 line

fastforward updates following offline_db rloadcat of r5429 and r5430 OVERRIDE
------------------------------------------------------------------------
r5430 | yuzy | 2013-03-19 09:28:48 +0800 (Tue, 19 Mar 2013) | 1 line

minor: Update offline_db ADScaled constants for MC with dybsvn:source:dybgaudi/trunk/Calibration/DBUpdate/UPDATES.txt@20070
------------------------------------------------------------------------
r5429 | beizhenhu | 2013-03-18 22:47:44 +0800 (Mon, 18 Mar 2013) | 1 line

dybsvn:source:dybgaudi/trunk/Calibration/DBUpdate/UPDATES.txt@20069
------------------------------------------------------------------------

Back from HEAD to before the yuzy commit:

[blyth@belle7 tmp_offline_db]$ svn update -r 5429
U    EnergyRecon/EnergyReconVld.csv
U    EnergyRecon/EnergyRecon.csv
U    CalibPmtFineGain/CalibPmtFineGainVld.csv
U    Reactor/ReactorVld.csv
U    Reactor/Reactor.csv
U    LOCALSEQNO/LOCALSEQNO.csv
Updated to revision 5429.
[blyth@belle7 tmp_offline_db]$

Merge yuzy commit into working copy:

[blyth@belle7 tmp_offline_db]$ svn merge http://dayabay.ihep.ac.cn/svn/dybaux/catalog/tmp_offline_db@5429 http://dayabay.ihep.ac.cn/svn/dybaux/catalog/tmp_offline_db@5430 .
--- Merging r5430 into '.':
U    EnergyRecon/EnergyReconVld.csv
U    EnergyRecon/EnergyRecon.csv
U    LOCALSEQNO/LOCALSEQNO.csv

[blyth@belle7 tmp_offline_db]$ svn st
M       EnergyRecon/EnergyReconVld.csv
M       EnergyRecon/EnergyRecon.csv
M       LOCALSEQNO/LOCALSEQNO.csv

[blyth@belle7 tmp_offline_db]$ dbsvn.py . -M     ## dbsvn approves the change
[blyth@belle7 tmp_offline_db]$ echo $?
0

[blyth@belle7 tmp_offline_db]$ dbsvn.py . -M -l debug
DEBUG:DybPython.dbsvn:starting with opts {'refcreds': '--username dayabay --password wrong', 'verbose': False, 'author': 'unknown', 'loglevel': 'debug', 'admins': '', 'no_message_chk': True, 'message': 'no-message', 'refpath': 'http://dayabay.ihep.ac.cn/svn/dybsvn', 'decoupled': False, 'revision': None}
DEBUG:DybPython.dbsvn:DBIValidate 46 lines of diff
DEBUG:DybPython.dbsvn:DBIValidate {'LOCALSEQNO': '-+', 'EnergyRecon': '+', 'EnergyReconVld': '+'}
[blyth@belle7 tmp_offline_db]$

Make unethical manual edit:

[blyth@belle7 tmp_offline_db]$ vi EnergyRecon/EnergyReconVld.csv
[blyth@belle7 tmp_offline_db]$ tail -2  EnergyRecon/EnergyReconVld.csv
2108,"2013-03-12 22:41:31","2038-01-19 03:14:07",4,2,3,0,-1,"2013-03-12 22:41:31","2013-03-19 01:12:07"
2109,"1970-01-01 00:00:00","2038-01-19 03:14:07",4,2,4,0,-1,"2013-03-12 22:41:31","2013-03-19 01:12:19"
##                                                 ^
##                                               SIMMASK

Now it squeals:

[blyth@belle7 tmp_offline_db]$ dbsvn.py . -M -l debug
DEBUG:DybPython.dbsvn:starting with opts {'refcreds': '--username dayabay --password wrong', 'verbose': False, 'author': 'unknown', 'loglevel': 'debug', 'admins': '', 'no_message_chk': True, 'message': 'no-message', 'refpath': 'http://dayabay.ihep.ac.cn/svn/dybsvn', 'decoupled': False, 'revision': None}
DEBUG:DybPython.dbsvn:DBIValidate 46 lines of diff
DEBUG:DybPython.dbsvn:DBIValidate {'LOCALSEQNO': '-+', 'EnergyRecon': '+', 'EnergyReconVld': '+'}
Traceback (most recent call last):
  File "/data1/env/local/dyb/NuWa-trunk/dybgaudi/InstallArea/scripts/dbsvn.py", line 4, in <module>
    main()
  File "/data1/env/local/dyb/NuWa-trunk/dybgaudi/InstallArea/python/DybPython/dbsvn.py", line 586, in main
    dbiv()
  File "/data1/env/local/dyb/NuWa-trunk/dybgaudi/InstallArea/python/DybPython/dbsvn.py", line 479, in __call__
    self.validate_validity()
  File "/data1/env/local/dyb/NuWa-trunk/dybgaudi/InstallArea/python/DybPython/dbsvn.py", line 432, in validate_validity
    self.validate_hunk(hunk)
  File "/data1/env/local/dyb/NuWa-trunk/dybgaudi/InstallArea/python/DybPython/dbsvn.py", line 451, in validate_hunk
    assert tbot < dt <= teot, ("time is out of range ", dt )
AssertionError: ('time is out of range ', datetime.datetime(1970, 1, 1, 0, 0))
[blyth@belle7 tmp_offline_db]$

[blyth@belle7 tmp_offline_db]$ which dbsvn.py            ## CAUTION NEED TO INSTALL DybPython after changes
/data1/env/local/dyb/NuWa-trunk/dybgaudi/InstallArea/scripts/dbsvn.py
[blyth@belle7 tmp_offline_db]$

Make changes to dbsvn.py and dbvld.py and install:

[blyth@belle7 DybPython]$ pwd
/data1/env/local/dyb/NuWa-trunk/dybgaudi/DybPython/python/DybPython
[blyth@belle7 DybPython]$ ( cd ../../cmt ; make DybPython_python )

Now it passes:

[blyth@belle7 tmp_offline_db]$ dbsvn.py . -M -l debug
DEBUG:DybPython.dbsvn:starting with opts {'refcreds': '--username dayabay --password wrong', 'verbose': False, 'author': 'unknown', 'loglevel': 'debug', 'admins': '', 'no_message_chk': True, 'message': 'no-message', 'refpath': 'http://dayabay.ihep.ac.cn/svn/dybsvn', 'decoupled': False, 'revision': None}
DEBUG:DybPython.dbsvn:DBIValidate 46 lines of diff
DEBUG:DybPython.dbsvn:DBIValidate {'LOCALSEQNO': '-+', 'EnergyRecon': '+', 'EnergyReconVld': '+'}
[blyth@belle7 tmp_offline_db]$ echo $?
0

Manual edit SIMMASK 2->1, and verify that it fails:

[blyth@belle7 tmp_offline_db]$ tail -1  EnergyRecon/EnergyReconVld.csv
2109,"1970-01-01 00:00:00","2038-01-19 03:14:07",4,1,4,0,-1,"2013-03-12 22:41:31","2013-03-19 01:12:19"