/search.css" rel="stylesheet" type="text/css"/> /search.js">
| Classes | Job Modules | Data Objects | Services | Algorithms | Tools | Packages | Directories | Tracs |

In This Package:

Public Member Functions | Public Attributes | Static Public Attributes | Properties | Private Member Functions
DybPython::dbicnf::DbiCnf Class Reference
Collaboration diagram for DybPython::dbicnf::DbiCnf:
Collaboration graph
[legend]

List of all members.

Public Member Functions

def argparser_
def __init__
def __call__
def __repr__
def parse_path
def writer
def logging_

Public Attributes

 irl
 for interactive testing
 args
 state for debug only

Static Public Attributes

string sites = "All|DayaBay|Far|LingAo|Mid|SAB"
 corresponds to an enum
string subsites = "AD1|AD2|AD3|AD4|All|IWS|OWS|RPC|Unknown"
string simflags = "Data|MC"

Properties

 defaults = property( _defaults )
 path = property( lambda self:self['path'] )
 dbno = property( lambda self:self['dbno'] )
 dummy = property( lambda self:self['dummy'] )
 runtimestart = property(lambda self:int(self['runtimestart']) if self['runtimestart'] else None)
 runtimeend = property(lambda self:int(self['runtimeend']) if self['runtimeend'] else None)
 timestart = property( _timestart, doc=_timestart.__doc__ )
 timeend = property( _timeend, doc=_timeend.__doc__ )
 site = property( _site )
 sitemask = property( _sitemask, doc=_sitemask.__doc__ )
 subsite = property( _subsite , doc=_subsite.__doc__)
 simflag = property( _simflag, doc=_simflag.__doc__ )
 simmask = property( _simmask, doc=_simmask.__doc__ )
 cr = property( _cr , doc=_cr.__doc__ )

Private Member Functions

def _defaults
def _irunlookup
def _timestart
def _timeend
def _site
def _sitemask
def _subsite
 bit icky
def _simflag
def _simmask
def _cr

Detailed Description

DbiCnf is a dict holding parameters that are inputs to defining 
the DBI writer and ingredients like contextrange etc..

All outputs of this class such as ``timestart``, ``cr`` etc..
are implemented as dynamically invoked properties, meaning 
that the only important state held is in this dict 
in the form of raw python types : str, int, datetime.

This dict is composed with class defaults, ctor arguments, commandline 
parsed results, path parameter regular expression parsed tokens, 
interactive updating.

Precedence in decreasing order:

#. commandline arguments
#. after ctor updates
#. ctor keyword arguments
#. basis defaults in ``DbiCnf.defaults``  

Usage in writer scripts::

   from DybPython import DbiCnf
   cnf = DbiCnf()
   cnf()              ## performs the parse  

   from DybDbi import GCalibPmtSpec, CSV
   wrt = cnf.writer( GCalibPmtSpec ) 
 
   src = CSV( cnf.path )
   for r in src:
       instance = GCalibPmtSpec.Create( **r )
       wrt.Write( instance )

   if not cnf.dummy:
       assert wrt.close()

Debugging/checking usage in ipython::

   from DybPython import DbiCnf
   cnf = DbiCnf(key=val,key2=val2)
   cnf['key3'] = 'val3'

   cnf()    ## performs command line parse
   cnf("All_AD1_Data.csv --task 20 --runtimestart 10 --dbconf tmp_offline_db:offline_db ")   ## test parsing gives desired params
   print cnf
   cnf['runtimestart'] = 10
   cnf.timestart            
   cnf['runtimestart'] = 1000    
   cnf.timestart                 ## will do timestart lookup for the changed run


The simplest and recommended usage is to define a standard `.csv` file naming convention.
For example when using the default context pattern::
   
   "^(?P<site>All|DayaBay|Far|LingAo|Mid|SAB)_(?P<subsite>AD1|AD2|AD3|AD4|All|IWS|OWS|RPC|Unknown)_(?P<simflag>MC|Data)\.csv"

The tokens `site`, `subsite` and `simflag` are extracted from basenames such as the below by the pattern matching.

#. SAB_AD1_Data.csv
#. SAB_AD2_Data.csv
    

Definition at line 23 of file dbicnf.py.


Constructor & Destructor Documentation

def DybPython::dbicnf::DbiCnf::__init__ (   self,
  args,
  kwa 
)
:param kwa: ctor keyword arguments override class defaults ``DbiCnf.defaults`` updating into `self` 

Definition at line 150 of file dbicnf.py.

00151                                      :
00152         """
00153         :param kwa: ctor keyword arguments override class defaults ``DbiCnf.defaults`` updating into `self` 
00154         """
00155         dict.__init__(self)
00156         self.update( self.defaults )  
00157         self.update( kwa )            
 

Member Function Documentation

def DybPython::dbicnf::DbiCnf::_defaults (   self) [private]

Definition at line 92 of file dbicnf.py.

00093                        :
00094         return dict( 
00095              loglevel="INFO", 
00096              logpath=None,
00097            logformat='%(asctime)s %(name)s %(levelname)-8s %(message)s',
00098                  task=0, 
00099                  site=None,
00100               subsite=None,
00101               simflag=None,
00102             timeformat="%Y-%m-%d %H:%M:%S", 
00103              timestart=None, 
00104                timeend=None,
00105             runtimestart=None, 
00106               runtimeend=None,
00107                   ctxptn="^(?P<site>%s)_(?P<subsite>%s)_(?P<simflag>%s)\.csv" % ( self.sites, self.subsites, self.simflags ) ,
00108                nomatch=False,
00109                dbconf="tmp_offline_db",
00110                   dbno=0, 
00111                dummy=False,
          )
def DybPython::dbicnf::DbiCnf::argparser_ (   self)

Definition at line 114 of file dbicnf.py.

00115                         :
00116         ap = argparse.ArgumentParser(description=__doc__, fromfile_prefix_chars='@', formatter_class=argparse.RawDescriptionHelpFormatter )
00117 
00118         po = ap.add_argument_group("positional")
00119         po.add_argument('path',                help='path to csv file, mandatory argument')
00120 
00121         op = ap.add_argument_group("operational")
00122         op.add_argument('-l','--loglevel',     help='logging level INFO,WARN,DEBUG... Default %(default)s ')
00123         op.add_argument(     "--logpath",      help="Path to write log file to. Default %(default)s ")
00124         op.add_argument(     "--logformat",    help="Used by logger. Default %(default)s ")
00125         op.add_argument('-n','--dummy',        help='Dummy run. Default %(default)s ', action="store_true" )
00126         
00127         ap.add_argument(     '--dbconf',       help='Section of ~/.my.cnf file. Default %(default)s when using runtimestart/runtimeend must be cascade such as tmp_offline_db:offline_db ')
00128 
00129         nu = ap.add_argument_group("not usually changed")
00130         nu.add_argument(     '--timeformat',   help='format for times. Default %(default)s ')
00131         nu.add_argument(     '--task',         help='set non zero for testing non-default algorithms. Default %(default)s ', type=int )
00132         nu.add_argument(     '--dbno',         help='DB number in cascade. Default %(default)s ', type=int )
00133 
00134         cx = ap.add_argument_group("context")
00135         cx.add_argument(     '--site',         help='string to be converted into enum integer with Site.FromString. Default %(default)s ', choices=self.sites.split("|"))
00136         cx.add_argument(     '--subsite',      help='string to be converted into enum integer with DetectorId.FromString. Default %(default)s ', choices=self.subsites.split("|"))
00137         cx.add_argument(     '--simflag',      help='string to be converted into enum integer with SimFlag.FromString. Default %(default)s ', choices=self.simflags.split("|"))
00138         cx.add_argument(     '--ctxptn',       help='Regular expression string to be matched against csv basenames. Often includes `site`, `subsite` and `simflag`. Default %(default)s ')
00139         cx.add_argument('-M','--nomatch',      help='Regular expression string is not forced to match csv basenames. Default %(default)s ', action="store_true" )
00140 
00141         ts = ap.add_mutually_exclusive_group()
00142         ts.add_argument(     '--timestart',    help='contextrange start time in UTC. Default %(default)s corresponds to TimeStamp.GetBOT  ', action=TimeAction )
00143         ts.add_argument(     '--runtimestart', help='contextrange start time in UTC corresponding to timestart for run number passed. Default %(default)s corresponds to use timestart ', type=int )
00144 
00145         te = ap.add_mutually_exclusive_group()
00146         te.add_argument(     '--timeend',      help='contextrange end time in UTC. Default %(default)s corresponds to TimeStamp.GetEOT ', action=TimeAction )
00147         te.add_argument(     '--runtimeend',   help='contextrange end time in UTC corresponding to timeend of run number passed. Default %(default)s corresponds to use timeend ', type=int )
00148 
00149         return ap 

def DybPython::dbicnf::DbiCnf::__call__ (   self,
  args_ = None 
)
:param args_: default of None parses system arguments, define for interactive testing

Performs parse steps:

#. populates argparser defaults from `self`
#. parses `args_` or sys.argv 
#. parses args.path obtaining ``pathdict``
#. updates `self` from args.path parsed tokens
#. updates `self` raw initial parsed values 

First positional argument is interpreted as a path and is pattern matched against 
a regular expression. 

Definition at line 158 of file dbicnf.py.

00159                                    :
00160         """
00161         :param args_: default of None parses system arguments, define for interactive testing
00162 
00163         Performs parse steps:
00164 
00165         #. populates argparser defaults from `self`
00166         #. parses `args_` or sys.argv 
00167         #. parses args.path obtaining ``pathdict``
00168         #. updates `self` from args.path parsed tokens
00169         #. updates `self` raw initial parsed values 
00170 
00171         First positional argument is interpreted as a path and is pattern matched against 
00172         a regular expression. 
00173         """
00174         ap = self.argparser_()
00175         ap.set_defaults( **self )         
00176 
00177         if args_ and type(args_) == str:
00178             args_ = shlex.split(args_)     ## for interactive testing 
00179 
00180         args = ap.parse_args(args=args_)
00181 
00182         logging.basicConfig( level=getattr(logging,args.loglevel.upper()) )
00183         log.info( "initial args %r " %  args )
00184         
00185         pathdict = self.parse_path( args.path , args.ctxptn, args.nomatch )
00186  
00187         self.update( pathdict )   
00188 
00189         va = vars(args)           
00190         for k,v in va.items(): 
00191             if not v:continue   ## None does not trump pre-existers
00192             self[k] = v 
00193 
00194         if self['dbconf']:
00195             os.environ['DBCONF'] = self['dbconf'] 
00196             log.warn("DBCONF set to %s " % self['dbconf'] )
00197 
00198         self.irl = self._irunlookup()
00199 
00200         ## state for debug only 
00201         self['pathdict'] = pathdict        
00202         self.args = args      

def DybPython::dbicnf::DbiCnf::__repr__ (   self)

Definition at line 203 of file dbicnf.py.

00204                       :
00205         return self.__class__.__name__ + "\n" + pformat(dict(self))

def DybPython::dbicnf::DbiCnf::_irunlookup (   self) [private]
**CAUTION** this takes the runtimestart/runtimeend directly from GDaqRunInfo lookup ...
so they must be UTC ... if that is not the case, that needs to be fixed 

Definition at line 206 of file dbicnf.py.

00207                          :
00208         """
00209         **CAUTION** this takes the runtimestart/runtimeend directly from GDaqRunInfo lookup ...
00210         so they must be UTC ... if that is not the case, that needs to be fixed 
00211         """
00212         rl = filter(None, (self['runtimestart'], self['runtimeend']))
00213         if rl:
00214             from DybDbi import IRunLookup
00215             log.debug("_irunlookup %r " % rl )
00216             irl = IRunLookup(*map(int,rl))
00217             log.debug("IRunLookup %r gives ... \n %s " % ( rl, pformat(irl) ))  
00218         else:
00219             irl = {}
00220         return irl 

def DybPython::dbicnf::DbiCnf::parse_path (   self,
  path_,
  ptn,
  nomatch 
)
Extract context metadata from the path using the regular expression string 
supplied. 

:param path: path to .csv source file
:param ptn: regular expression string that can contain tokens for any config parameters

:rtype dict: dict of strings extracted from the path 

Definition at line 221 of file dbicnf.py.

00222                                                 :
00223         """
00224         Extract context metadata from the path using the regular expression string 
00225         supplied. 
00226 
00227         :param path: path to .csv source file
00228         :param ptn: regular expression string that can contain tokens for any config parameters
00229 
00230         :rtype dict: dict of strings extracted from the path 
00231         """ 
00232         path = os.path.expandvars(os.path.expanduser(path_))  ## expand envvars or twiddles
00233         name = os.path.basename( path )
00234         ptn_ = re.compile(ptn)
00235         match = ptn_.match(  name )
00236         pathdict = {}
00237         if match:
00238             pathdict = match.groupdict()
00239         else:
00240             if nomatch:
00241                 log.warning("did not match name %s with pattern %s " % ( name , ptn ) )
00242             else:
00243                 log.fatal("failed to match name %s with pattern %s " % ( name , ptn ) )
00244                 raise Exception
00245         log.debug( "pathdict %r " %  pathdict )
00246         return pathdict 
00247 

def DybPython::dbicnf::DbiCnf::_timestart (   self) [private]

Definition at line 256 of file dbicnf.py.

00257                         :
00258         """
00259         """
00260         from DybDbi import TimeStamp
00261         rts = self.runtimestart
00262         if rts:
00263             if not rts in self.irl:
00264                 self._irunlookup()
00265             timestart = self.irl[rts].vrec.contextrange.timestart
00266         else:
00267             timestart = self['timestart']
00268             timestart = TimeStamp.fromAssumedUTCDatetime( timestart ) if timestart else TimeStamp.GetBOT()
        return timestart
def DybPython::dbicnf::DbiCnf::_timeend (   self) [private]

Definition at line 271 of file dbicnf.py.

00272                       :
00273         """
00274         """
00275         from DybDbi import TimeStamp
00276         rte = self.runtimeend 
00277         if rte:
00278             if not rte in self.irl:
00279                 self._irunlookup()
00280             timeend = self.irl[rte].vrec.contextrange.timeend
00281         else:
00282             timeend   = self['timeend']
00283             timeend = TimeStamp.fromAssumedUTCDatetime( timeend ) if timeend else TimeStamp.GetEOT()
        return timeend  
def DybPython::dbicnf::DbiCnf::_site (   self) [private]
Convert string site into enum integer

Definition at line 287 of file dbicnf.py.

00288                    :
00289         """Convert string site into enum integer"""
00290         from DybDbi import Site
        return Site.FromString( self['site'] )
def DybPython::dbicnf::DbiCnf::_sitemask (   self) [private]
Convert string site into enum integer
   if multi-site masks are needed will have to revisit this

Definition at line 292 of file dbicnf.py.

00293                        :
00294         """Convert string site into enum integer
00295            if multi-site masks are needed will have to revisit this
00296         """
00297         from DybDbi import Site
        return Site.FromString( self['site'] )
def DybPython::dbicnf::DbiCnf::_subsite (   self) [private]

bit icky

Convert string subsite/DetectorId into enum integer

Definition at line 300 of file dbicnf.py.

00301                       :
00302         """Convert string subsite/DetectorId into enum integer"""
00303         from DybDbi import DetectorId
        return DetectorId.FromString( self['subsite'] )
def DybPython::dbicnf::DbiCnf::_simflag (   self) [private]
Convert string simflag into enum integer

Definition at line 306 of file dbicnf.py.

00307                       :
00308         """Convert string simflag into enum integer"""
00309         from DybDbi import SimFlag
        return SimFlag.FromString( self['simflag'] )
def DybPython::dbicnf::DbiCnf::_simmask (   self) [private]
Convert string simflag into enum integer
   (note the simflag is interpreted as the mask)

Definition at line 311 of file dbicnf.py.

00312                       :
00313         """Convert string simflag into enum integer
00314            (note the simflag is interpreted as the mask)
00315         """
00316         from DybDbi import SimFlag
        return SimFlag.FromString( self['simflag'] )
def DybPython::dbicnf::DbiCnf::_cr (   self) [private]
Convert the strings into enum value, and datetimes into TimeStamps in
order to create the ContextRange instance 

:return: context range instance 

Definition at line 319 of file dbicnf.py.

00320                  :
00321         """
00322         Convert the strings into enum value, and datetimes into TimeStamps in
00323         order to create the ContextRange instance 
00324 
00325         :return: context range instance 
00326         """ 
00327         from DybDbi import ContextRange
        return ContextRange( self.sitemask, self.simmask, self.timestart, self.timeend )
def DybPython::dbicnf::DbiCnf::writer (   self,
  kls 
)
Create a pre-configured DybDbi writer based on 
arguments and source csv filename parsing and 
creates the corresponding DB table if it does not exist.

:param kls: DybDbi class, eg GCalibPmtHighGain

Definition at line 335 of file dbicnf.py.

00336                           :
00337         """
00338         Create a pre-configured DybDbi writer based on 
00339         arguments and source csv filename parsing and 
00340         creates the corresponding DB table if it does not exist.
00341 
00342         :param kls: DybDbi class, eg GCalibPmtHighGain
00343 
00344         """
00345         from DybDbi import TimeStamp
00346 
00347         assert kls.__name__[0] == 'G', "DybDbi classes must start with 'G' unlike %s " % kls.__name__
00348         name = kls.__name__[1:]
00349         assert self.dbno == 0 , "unexpected dbno %s " % self.dbno
00350 
00351         log.info("creating DB tables for %s in dbno %s  " % ( name , self.dbno )) 
00352         kls().CreateDatabaseTables( self.dbno , name )
00353 
00354         wrt = kls.Wrt() 
00355         wrt.ctx( 
00356              contextrange=self.cr, 
00357              dbno=self.dbno ,
00358              versiondate=TimeStamp(0,0),       
00359              subsite=self.subsite,
00360         )
00361         return wrt
00362 

def DybPython::dbicnf::DbiCnf::logging_ (   self,
  args 
)
Hmm need some work ...

Definition at line 363 of file dbicnf.py.

00364                             :
00365         """
00366         Hmm need some work ...
00367         """
00368         loglevel = getattr(logging,args.loglevel.upper())
00369 
00370         sh = logging.StreamHandler()
00371         sh.setLevel(loglevel)
00372         fmtr = logging.Formatter(args.logformat)
00373         sh.setFormatter(fmtr)
00374         log.addHandler(sh)
00375 
00376         if args.logpath:
00377             fh = logging.FileHandler(args.logpath,mode="w")
00378             fh.setFormatter(fmtr)
00379             fh.setLevel(loglevel)
00380             log.addHandler(fh)
00381 
00382 
00383 


Member Data Documentation

string DybPython::dbicnf::DbiCnf::sites = "All|DayaBay|Far|LingAo|Mid|SAB" [static]

corresponds to an enum

Definition at line 88 of file dbicnf.py.

string DybPython::dbicnf::DbiCnf::subsites = "AD1|AD2|AD3|AD4|All|IWS|OWS|RPC|Unknown" [static]

Definition at line 89 of file dbicnf.py.

string DybPython::dbicnf::DbiCnf::simflags = "Data|MC" [static]

Definition at line 90 of file dbicnf.py.

for interactive testing

None does not trump pre-existers

Definition at line 173 of file dbicnf.py.

state for debug only

Definition at line 174 of file dbicnf.py.


Property Documentation

DybPython::dbicnf::DbiCnf::defaults = property( _defaults ) [static]

Definition at line 112 of file dbicnf.py.

DybPython::dbicnf::DbiCnf::path = property( lambda self:self['path'] ) [static]

Definition at line 249 of file dbicnf.py.

DybPython::dbicnf::DbiCnf::dbno = property( lambda self:self['dbno'] ) [static]

Definition at line 250 of file dbicnf.py.

DybPython::dbicnf::DbiCnf::dummy = property( lambda self:self['dummy'] ) [static]

Definition at line 251 of file dbicnf.py.

DybPython::dbicnf::DbiCnf::runtimestart = property(lambda self:int(self['runtimestart']) if self['runtimestart'] else None) [static]

Definition at line 253 of file dbicnf.py.

DybPython::dbicnf::DbiCnf::runtimeend = property(lambda self:int(self['runtimeend']) if self['runtimeend'] else None) [static]

Definition at line 254 of file dbicnf.py.

DybPython::dbicnf::DbiCnf::timestart = property( _timestart, doc=_timestart.__doc__ ) [static]

Definition at line 269 of file dbicnf.py.

DybPython::dbicnf::DbiCnf::timeend = property( _timeend, doc=_timeend.__doc__ ) [static]

Definition at line 284 of file dbicnf.py.

DybPython::dbicnf::DbiCnf::site = property( _site ) [static]

Definition at line 291 of file dbicnf.py.

DybPython::dbicnf::DbiCnf::sitemask = property( _sitemask, doc=_sitemask.__doc__ ) [static]

Definition at line 298 of file dbicnf.py.

DybPython::dbicnf::DbiCnf::subsite = property( _subsite , doc=_subsite.__doc__) [static]

Definition at line 304 of file dbicnf.py.

DybPython::dbicnf::DbiCnf::simflag = property( _simflag, doc=_simflag.__doc__ ) [static]

Definition at line 310 of file dbicnf.py.

DybPython::dbicnf::DbiCnf::simmask = property( _simmask, doc=_simmask.__doc__ ) [static]

Definition at line 317 of file dbicnf.py.

DybPython::dbicnf::DbiCnf::cr = property( _cr , doc=_cr.__doc__ ) [static]

Definition at line 328 of file dbicnf.py.


The documentation for this class was generated from the following file:
| Classes | Job Modules | Data Objects | Services | Algorithms | Tools | Packages | Directories | Tracs |

Generated on Fri May 16 2014 09:55:40 for DybPython by doxygen 1.7.4