Other api/dybpython

Table Of Contents

Previous topic


Next topic


This Page

Daya Bay Links

Content Skeleton


turns python/DybPython/ into a Python package


class DybPython.Control.NuWa[source]

This is the main program to run NuWa offline jobs.

It provides a job with a minimal, standard setup. Non standard behavior can made using command line options or providing additional configuration in the form of python files or modules to load.


nuwa.py --help 
nuwa.py [options] [-m|--module "mod.ule --mod-arg ..."] \
        [config1.py config2.py ...] \
        [mod.ule1 mod.ule2 ...] \
        [[input1.root input2.root ...] or [input1.data ...]] \

Python modules can be specified with -m|–module options and may include any per-module arguments by enclosing them in shell quotes as in the above usage. Modules that do not take arguments may also be listed as non-option arguments. Modules may supply the following functions:

  1. configure(argv=[]) - if exists, executed at configuration time
  2. run(theApp) - if exists, executed at run time with theApp set to the AppMgr.

Additionally, python job scripts may be specified.

Modules and scripts are loaded in the order they are specified on the command line.

Finally, input ROOT files may be specified. These will be read in the order they are specified and will be assigned to supplying streams not specificially specified in any input-stream map.

The listing of modules, job scripts and/or ROOT files may be interspersed but must follow all options.

In addition to the command line, arguments can be given in a text file with one line per argument. This file can then be given to nuwa.py on the command line prefaced with an ‘@’ or a ‘+’.

Create a NuWa instance.


Add file name or list of file names to self.input_files, expanding if it is a .list file.

add_service_redirect(alias, name)[source]

Make alias an alias for given service. Should be called during configuration only


Parse command line


spin over all non-option arguments


Existance of DBCONF envvar is used as a signal to switch between Static and DB services, so pull it out separate for clarity


For motivation for DbiSvc level configuration, see dybsvn:ticket:842


Configure common Dyb services


Set up framework level defaults


If ipython not available or are already inside ipython, setup a dummy embedded ipython ipshell function, otherwise setup the real thing.

configure_mod(modname, modargs=None)[source]

Configure this module, add to job


load and configure() “-m” modules here


Set up python features


Configure for “quanjing/panoramix” visualization


Return True if file name has a recognized extension.


Run time addition of Python Algs so they are in correct module-order


An example using commandline parsing and pattern match against filenames, allowing smart DBI writer scripts to be created that minimize code duplication.

However make sure that arguments used are still captured into the repository either by creating one line scripts that invoke the flexible scripts. Or arranging for flexible scripts to read driver files.

class DybPython.dbicnf.DbiCnf(*args, **kwa)[source]

Bases: dict

DbiCnf is a dict holding parameters that are inputs to defining the DBI writer and ingredients like contextrange etc..

All outputs of this class such as timestart, cr etc.. are implemented as dynamically invoked properties, meaning that the only important state held is in this dict in the form of raw python types : str, int, datetime.

This dict is composed with class defaults, ctor arguments, commandline parsed results, path parameter regular expression parsed tokens, interactive updating.

Precedence in decreasing order:

  1. commandline arguments
  2. after ctor updates
  3. ctor keyword arguments
  4. basis defaults in DbiCnf.defaults

Usage in writer scripts:

from DybPython import DbiCnf
cnf = DbiCnf()
cnf()              ## performs the parse  

from DybDbi import GCalibPmtSpec, CSV
wrt = cnf.writer( GCalibPmtSpec ) 

src = CSV( cnf.path )
for r in src:
    instance = GCalibPmtSpec.Create( **r )
    wrt.Write( instance )

if not cnf.dummy:
    assert wrt.close()

Debugging/checking usage in ipython:

from DybPython import DbiCnf
cnf = DbiCnf(key=val,key2=val2)
cnf['key3'] = 'val3'

cnf()    ## performs command line parse
cnf("All_AD1_Data.csv --task 20 --runtimestart 10 --dbconf tmp_offline_db:offline_db ")   ## test parsing gives desired params
print cnf
cnf['runtimestart'] = 10
cnf['runtimestart'] = 1000    
cnf.timestart                 ## will do timestart lookup for the changed run

The simplest and recommended usage is to define a standard .csv file naming convention. For example when using the default context pattern:


The tokens site, subsite and simflag are extracted from basenames such as the below by the pattern matching.

  1. SAB_AD1_Data.csv
  2. SAB_AD2_Data.csv
Parameters:kwa – ctor keyword arguments override class defaults DbiCnf.defaults updating into self

Convert the strings into enum value, and datetimes into TimeStamps in order to create the ContextRange instance

Returns:context range instance

Hmm need some work ...

parse_path(path_, ptn, nomatch)[source]

Extract context metadata from the path using the regular expression string supplied.

  • path – path to .csv source file
  • ptn – regular expression string that can contain tokens for any config parameters
Rtype dict:

dict of strings extracted from the path


Convert string simflag into enum integer


Convert string simflag into enum integer (note the simflag is interpreted as the mask)


Convert string site into enum integer


Convert string site into enum integer if multi-site masks are needed will have to revisit this


Convert string subsite/DetectorId into enum integer


Create a pre-configured DybDbi writer based on arguments and source csv filename parsing and creates the corresponding DB table if it does not exist.

Parameters:kls – DybDbi class, eg GCalibPmtHighGain
class DybPython.dbicnf.TimeAction(option_strings, dest, nargs=None, const=None, default=None, type=None, choices=None, required=False, help=None, metavar=None)[source]

Bases: argparse.Action

Converts string date representations into datetimes