Other api/dybpython

Table Of Contents

Previous topic

DybDbi.GDcsPmtHv

Next topic

DbiDataSvc

This Page

Daya Bay Links

Content Skeleton

DybPython

turns python/DybPython/ into a Python package

DybPython.Control

class DybPython.Control.NuWa[source]

This is the main program to run NuWa offline jobs.

It provides a job with a minimal, standard setup. Non standard behavior can made using command line options or providing additional configuration in the form of python files or modules to load.

Usage:

nuwa.py --help 
nuwa.py [options] [-m|--module "mod.ule --mod-arg ..."] \
        [config1.py config2.py ...] \
        [mod.ule1 mod.ule2 ...] \
        [[input1.root input2.root ...] or [input1.data ...]] \

Python modules can be specified with -m|–module options and may include any per-module arguments by enclosing them in shell quotes as in the above usage. Modules that do not take arguments may also be listed as non-option arguments. Modules may supply the following functions:

  1. configure(argv=[]) - if exists, executed at configuration time
  2. run(theApp) - if exists, executed at run time with theApp set to the AppMgr.

Additionally, python job scripts may be specified.

Modules and scripts are loaded in the order they are specified on the command line.

Finally, input ROOT files may be specified. These will be read in the order they are specified and will be assigned to supplying streams not specificially specified in any input-stream map.

The listing of modules, job scripts and/or ROOT files may be interspersed but must follow all options.

In addition to the command line, arguments can be given in a text file with one line per argument. This file can then be given to nuwa.py on the command line prefaced with an ‘@’ or a ‘+’.

Create a NuWa instance.

add_input_file(fname)[source]

Add file name or list of file names to self.input_files, expanding if it is a .list file.

add_service_redirect(alias, name)[source]

Make alias an alias for given service. Should be called during configuration only

cmdline(argv)[source]

Parse command line

configure_args()[source]

spin over all non-option arguments

configure_dbconf()[source]

Existance of DBCONF envvar is used as a signal to switch between Static and DB services, so pull it out separate for clarity

configure_dbi()[source]

For motivation for DbiSvc level configuration, see dybsvn:ticket:842

configure_dyb_services()[source]

Configure common Dyb services

configure_framework()[source]

Set up framework level defaults

configure_ipython()[source]

If ipython not available or are already inside ipython, setup a dummy embedded ipython ipshell function, otherwise setup the real thing.

configure_mod(modname, modargs=None)[source]

Configure this module, add to job

configure_optmods()[source]

load and configure() “-m” modules here

configure_python_features()[source]

Set up python features

configure_visualization()[source]

Configure for “quanjing/panoramix” visualization

known_input_type(fname)[source]

Return True if file name has a recognized extension.

run_post_user(appMgr)[source]

Run time addition of Python Algs so they are in correct module-order

DybPython.dbicnf

An example using commandline parsing and pattern match against filenames, allowing smart DBI writer scripts to be created that minimize code duplication.

However make sure that arguments used are still captured into the repository either by creating one line scripts that invoke the flexible scripts. Or arranging for flexible scripts to read driver files.

class DybPython.dbicnf.DbiCnf(*args, **kwa)[source]

Bases: dict

DbiCnf is a dict holding parameters that are inputs to defining the DBI writer and ingredients like contextrange etc..

All outputs of this class such as timestart, cr etc.. are implemented as dynamically invoked properties, meaning that the only important state held is in this dict in the form of raw python types : str, int, datetime.

This dict is composed with class defaults, ctor arguments, commandline parsed results, path parameter regular expression parsed tokens, interactive updating.

Precedence in decreasing order:

  1. commandline arguments
  2. after ctor updates
  3. ctor keyword arguments
  4. basis defaults in DbiCnf.defaults

Usage in writer scripts:

from DybPython import DbiCnf
cnf = DbiCnf()
cnf()              ## performs the parse  

from DybDbi import GCalibPmtSpec, CSV
wrt = cnf.writer( GCalibPmtSpec ) 

src = CSV( cnf.path )
for r in src:
    instance = GCalibPmtSpec.Create( **r )
    wrt.Write( instance )

if not cnf.dummy:
    assert wrt.close()

Debugging/checking usage in ipython:

from DybPython import DbiCnf
cnf = DbiCnf(key=val,key2=val2)
cnf['key3'] = 'val3'

cnf()    ## performs command line parse
cnf("All_AD1_Data.csv --task 20 --runtimestart 10 --dbconf tmp_offline_db:offline_db ")   ## test parsing gives desired params
print cnf
cnf['runtimestart'] = 10
cnf.timestart            
cnf['runtimestart'] = 1000    
cnf.timestart                 ## will do timestart lookup for the changed run

The simplest and recommended usage is to define a standard .csv file naming convention. For example when using the default context pattern:

"^(?P<site>All|DayaBay|Far|LingAo|Mid|SAB)_(?P<subsite>AD1|AD2|AD3|AD4|All|IWS|OWS|RPC|Unknown)_(?P<simflag>MC|Data)\.csv"

The tokens site, subsite and simflag are extracted from basenames such as the below by the pattern matching.

  1. SAB_AD1_Data.csv
  2. SAB_AD2_Data.csv
Parameters:kwa – ctor keyword arguments override class defaults DbiCnf.defaults updating into self
cr

Convert the strings into enum value, and datetimes into TimeStamps in order to create the ContextRange instance

Returns:context range instance
logging_(args)[source]

Hmm need some work ...

parse_path(path_, ptn, nomatch)[source]

Extract context metadata from the path using the regular expression string supplied.

Parameters:
  • path – path to .csv source file
  • ptn – regular expression string that can contain tokens for any config parameters
Rtype dict:

dict of strings extracted from the path

simflag

Convert string simflag into enum integer

simmask

Convert string simflag into enum integer (note the simflag is interpreted as the mask)

site

Convert string site into enum integer

sitemask

Convert string site into enum integer if multi-site masks are needed will have to revisit this

subsite

Convert string subsite/DetectorId into enum integer

timeend
timestart
writer(kls)[source]

Create a pre-configured DybDbi writer based on arguments and source csv filename parsing and creates the corresponding DB table if it does not exist.

Parameters:kls – DybDbi class, eg GCalibPmtHighGain
class DybPython.dbicnf.TimeAction(option_strings, dest, nargs=None, const=None, default=None, type=None, choices=None, required=False, help=None, metavar=None)[source]

Bases: argparse.Action

Converts string date representations into datetimes