Other sop/dbdebug

Table Of Contents

Previous topic

DB Table Reading

Next topic

DB Table Creation

This Page

Daya Bay Links

Content Skeleton

Debugging unexpected parameters

The key to debugging is isolation of issues. The recommended first steps to locate where problems are:

Configuration check

When operating at the service level first verify that you are using the desired services, ie that you are not using StaticCalibDataSvc when you intend to use the DB with DbiCalibDataSvc. An example of switching services is provided by dybsvn:r11843

Verify that you are connecting to the DB you expect. Avoid confusing config such as having multiple updates of DBCONF

[blyth@belle7 dbtest]$ grep DBCONF \*.py
runCalib.py:os.environ['DBCONF'] = "offline_db"
testCalibDirectly.py:os.environ.update( DBCONF="tmp_wangzm_offline_db" )

For reading use of os.environ.setdefault("DBCONF", "offline_db") is recommended, allowing external overriding.

ipython DybDbi session

Get into ipython either with DBCONF set externally or internally. Adjust the default context (for example to correspond to the timestamp of a data file) and do a DBI query:

In [1]: import os

In [1]: os.environ['DBCONF'] = "tmp_offline_db"

In [1]: from DybDbi import GCalibPmtSpec, TimeStamp

In [2]: rpt = GCalibPmtSpec.Rpt()

In [3]: from DybDbi import TimeStamp

In [4]: rpt.ctx(timestamp=TimeStamp(2011, 1, 22, 10,0,0 ) )

In [5]: len(rpt)

In [6]: rpt[0]

DbiDataSvc test

Try running the standard DbiDataSvc.TestDbiDataSvc with an appropriate timestamp.

DBCONF="tmp_offline_db" nuwa.py -A none --history=off -n 1 -m "DbiDataSvc.TestDbiDataSvc --timeString=2011-1-24-0-0-0"

Note

additional context flexibility for this tool would improve it’s usefulness as a debugging tool

mysql client session

Check your [client] section is pointed to the same DB and perform some simple queries:

mysql> select * from CalibPmtSpecVld order by TIMESTART ;
+-------+---------------------+---------------------+----------+---------+---------+------+-------------+---------------------+---------------------+
| SEQNO | TIMESTART           | TIMEEND             | SITEMASK | SIMMASK | SUBSITE | TASK | AGGREGATENO | VERSIONDATE         | INSERTDATE          |
+-------+---------------------+---------------------+----------+---------+---------+------+-------------+---------------------+---------------------+
|    31 | 1970-01-01 00:00:00 | 2038-01-19 03:14:07 |      127 |       3 |       0 |    7 |          -1 | 1970-01-01 00:01:00 | 2011-04-25 07:53:19 |
|    30 | 1970-01-01 00:00:00 | 2038-01-19 03:14:07 |      127 |       3 |       0 |    7 |          -1 | 1970-01-01 00:00:00 | 2011-04-25 07:00:59 |
|    18 | 2010-06-21 07:49:24 | 2038-01-19 03:14:07 |       32 |       1 |       1 |    0 |          -1 | 2010-06-21 15:50:24 | 2010-07-19 12:49:29 |
|    23 | 2010-09-16 06:31:34 | 2038-01-19 03:14:07 |       32 |       1 |       1 |    0 |          -1 | 2010-06-21 15:55:24 | 2010-10-13 14:23:35 |
|    24 | 2010-09-21 05:48:57 | 2038-01-19 03:14:07 |       32 |       1 |       2 |    0 |          -1 | 2010-06-21 15:56:24 | 2010-10-13 14:24:44 |
|    25 | 2010-09-22 04:26:59 | 2038-01-19 03:14:07 |       32 |       1 |       2 |    0 |          -1 | 2010-06-21 15:57:24 | 2010-10-13 14:25:32 |
|    29 | 2011-01-22 08:15:17 | 2038-01-19 03:14:07 |       32 |       1 |       1 |    0 |          -1 | 2010-06-21 15:56:24 | 2011-02-28 15:02:13 |
|    28 | 2011-01-22 08:15:17 | 2038-01-19 03:14:07 |        1 |       2 |       1 |    0 |          -1 | 2011-01-22 08:15:17 | 2011-02-28 10:47:15 |
|    27 | 2011-01-22 08:15:17 | 2020-12-30 16:00:00 |      127 |       2 |       0 |    0 |          -1 | 2011-01-22 08:15:17 | 2011-02-25 08:28:04 |
|    26 | 2011-01-22 08:15:17 | 2020-12-30 16:00:00 |      127 |       1 |       0 |    0 |          -1 | 2011-01-22 08:15:17 | 2011-02-25 08:10:15 |
+-------+---------------------+---------------------+----------+---------+---------+------+-------------+---------------------+---------------------+
10 rows in set (0.00 sec)

Following the mysql tail

If you have access to the DB server machine and have privileges to access the mysql log file it is exceedingly informative to leave a process tailing the mysql log. For example with:

sudo tail -f /var/log/mysql.log

This allows observation of the mysql commands performed as you interactively make queries from ipython:

[blyth@belle7 ~]$ mysql-tail
                  64352 Query       SHOW COLUMNS FROM `CalibPmtSpecVld`
                  64352 Query       SHOW TABLE STATUS LIKE 'CalibPmtSpecVld'
                  64352 Prepare     [1] select * from CalibPmtSpecVld where     TimeStart <= '2011-02-01 10:00:00' and TimeEnd    > '2011-01-12 10:00:00' and SiteMask & 127 and SimMask & 1 and  Task = 0 and SubSite = 0 order by VERSIONDATE desc
                  64352 Execute     [1] select * from CalibPmtSpecVld where     TimeStart <= '2011-02-01 10:00:00' and TimeEnd    > '2011-01-12 10:00:00' and SiteMask & 127 and SimMask & 1 and  Task = 0 and SubSite = 0 order by VERSIONDATE desc
                  64352 Prepare     [2] select min(TIMESTART) from CalibPmtSpecVld where TIMESTART > '2011-02-01 10:00:00'  and SiteMask & 127 and SimMask & 1 and VERSIONDATE >= '2011-01-22 08:15:17' and  SubSite = 0 and  Task = 0
                  64352 Execute     [2] select min(TIMESTART) from CalibPmtSpecVld where TIMESTART > '2011-02-01 10:00:00'  and SiteMask & 127 and SimMask & 1 and VERSIONDATE >= '2011-01-22 08:15:17' and  SubSite = 0 and  Task = 0
                  64352 Prepare     [3] select min(TIMEEND) from CalibPmtSpecVld where TIMEEND > '2011-02-01 10:00:00'  and SiteMask & 127 and SimMask & 1 and VERSIONDATE >= '2011-01-22 08:15:17' and  SubSite = 0 and  Task = 0
                  64352 Execute     [3] select min(TIMEEND) from CalibPmtSpecVld where TIMEEND > '2011-02-01 10:00:00'  and SiteMask & 127 and SimMask & 1 and VERSIONDATE >= '2011-01-22 08:15:17' and  SubSite = 0 and  Task = 0
                  64352 Prepare     [4] select max(TIMESTART) from CalibPmtSpecVld where TIMESTART < '2011-01-12 10:00:00'  and SiteMask & 127 and SimMask & 1 and VERSIONDATE >= '2011-01-22 08:15:17' and  SubSite = 0 and  Task = 0
                  64352 Execute     [4] select max(TIMESTART) from CalibPmtSpecVld where TIMESTART < '2011-01-12 10:00:00'  and SiteMask & 127 and SimMask & 1 and VERSIONDATE >= '2011-01-22 08:15:17' and  SubSite = 0 and  Task = 0
                  64352 Prepare     [5] select max(TIMEEND) from CalibPmtSpecVld where TIMEEND < '2011-01-12 10:00:00'  and SiteMask & 127 and SimMask & 1 and VERSIONDATE >= '2011-01-22 08:15:17' and  SubSite = 0 and  Task = 0
                  64352 Execute     [5] select max(TIMEEND) from CalibPmtSpecVld where TIMEEND < '2011-01-12 10:00:00'  and SiteMask & 127 and SimMask & 1 and VERSIONDATE >= '2011-01-22 08:15:17' and  SubSite = 0 and  Task = 0
                  64352 Prepare     [6] select * from CalibPmtSpec where     SEQNO= 26
                  64352 Execute     [6] select * from CalibPmtSpec where     SEQNO= 26
                  64352 Quit

The SQL queries from the log can then be copy-and-pasted to a mysql client session for interactive examination.

Todo

Provide a way for non-administrators to do this style of debugging, perhaps with an extra DBI log file ?

GDB Debugging of Template Laden DBI

Isolate issue into small python test, then:

gdb $(which python)
(gdb) set args test_dybdbi_write.py
(gdb) b "DbiWriter<GDcsPmtHv>::operator<<(GDcsPmtHv const&)"
Function "DbiWriter<GDcsPmtHv>::operator<<(GDcsPmtHv const&)" not defined.
Make breakpoint pending on future shared library load? (y or [n]) y
(gdb) r

Determining the symbol names (especially when templated) can be painful.:

(gdb) b "DbiWriter<GDcsPmtHv>&
Display all 378426 possibilities? (y or n)
(gdb) b DbiWriter.tpl:100

Hunting for symbols inside the .so much is faster than attempting to use tab completion, or manually compose the magic symbol names:

objdump -t ./i686-slc4-gcc34-dbg/libDybDbiLib.so | c++filt | grep DbiWriter\<GDcsPmtHv\>::operator
0014b0ae  w    F .text  000002ca              DbiWriter<GDcsPmtHv>::operator<<(GDcsPmtHv const&)