/search.css" rel="stylesheet" type="text/css"/> /search.js">
| Classes | Job Modules | Data Objects | Services | Algorithms | Tools | Packages | Directories | Tracs |

In This Package:

Public Member Functions | Public Attributes | Static Public Attributes | Properties | Private Member Functions | Static Private Attributes
DybPython::dbconf::DBConf Class Reference
Collaboration diagram for DybPython::dbconf::DBConf:
Collaboration graph
[legend]

List of all members.

Public Member Functions

def Export
def __init__
def mysqldb_parameters
def configure_cascade
def dump_env
def smry
def from_env
def export_
def read_cfg
def has_config
def prime_parser
def export_to_env

Public Attributes

 secure
 verbose
 sect
 first or only
 path
 user
 pswd
 url
 host
 db
 fix
 fixpass
 nodb
 cascade
 zeroth slot
 export

Static Public Attributes

dictionary defaults
tuple Export = classmethod( Export )
tuple from_env = classmethod(from_env)
tuple read_cfg = classmethod( read_cfg )
tuple has_config = classmethod( has_config )
tuple prime_parser = classmethod( prime_parser )

Properties

 urls = property( lambda self:(self.url % self).split(";") )
 users = property( lambda self:(self.user % self).split(";") )
 pswds = property( lambda self:(self.pswd % self).split(";") )
 fixs = property( lambda self:(self.fix % self).split(";") )

Private Member Functions

def _check_path

Static Private Attributes

string __str__ = "\n"

Detailed Description

Reads a section of the Database configuration file, 
storing key/value pairs into this dict. The default file *path* is ``~/.my.cnf``
which is formatted like::

   [testdb]
   host      = dybdb1.ihep.ac.cn
   database  = testdb
   user      = dayabay
   password  = youknowoit

The standard python :py:mod:`ConfigParser` is used, 
which supports ``%(name)s`` style replacements in other values. 
 
Usage example::

   from DybPython import DBConf
   dbc = DBConf(sect="client", path="~/.my.cnf" ) 
   print dbc['host']

   dbo = DBConf("offline_db")
   assert dbo['host'] == "dybdb1.ihep.ac.cn"

.. warning::
     As passwords are contained **DO NOT COMMIT** into any repository, and protect the file.

Definition at line 13 of file dbconf.py.


Constructor & Destructor Documentation

def DybPython::dbconf::DBConf::__init__ (   self,
  sect = None,
  path = None,
  user = None,
  pswd = None,
  url = None,
  host = None,
  db = None,
  fix = None,
  fixpass = None,
  restrict = None,
  verbose = False,
  secure = False,
  from_env = False,
  nodb = False 
)

See also :ref:`dbi:running` section of the Offline User Manual 

Interpolates the DB connection parameter patterns gleaned 
from arguments, envvars or defaults (in that precedence order)
into usable values using the context supplied by the 
*sect* section of the ini format config file at *path*

Optional keyword arguments:
     
       ================   =======================================================
       Keyword            Description
       ================   =======================================================
  *sect*            section in config file 
  *path*            colon delimited list of paths to config file

  *user*            username 
  *pswd*            password
  *url*             connection url
  *host*            db host 
  *db*              db name

  *fix*             triggers fixture loading into temporary 
                    spawned cascade and specifies paths to fixture files
                    for each member of the cascade (semi-colon delimited)  
  *fixpass*         skip the DB cascade  dropping/creation that is 
                    normally done
                    as part of cascade spawning (used in DBWriter/tests) 
  *restrict*        constrain the names of DB that can connect to 
                    starting with a string, eg ``tmp_`` as a safeguard
  *nodb*            used to connect without specifying the database
                    this requires greater access privileges and is used
                    to perform database dropping/creation
       ================   =======================================================

                    
Correspondingly named envvars can also be used:

.. envvar:: DBCONF 
.. envvar:: DBCONF_PATH  

.. envvar:: DBCONF_USER
.. envvar:: DBCONF_PWSD
.. envvar:: DBCONF_URL
.. envvar:: DBCONF_HOST
.. envvar:: DBCONF_DB

.. envvar:: DBCONF_FIX
.. envvar:: DBCONF_FIXPASS
.. envvar:: DBCONF_RESTRICT

The :envvar:`DBCONF` existance also triggers the 
:meth:`DybPython.dbconf.DBConf.Export` in :dybgaudi:`Database/DatabaseInterface/src/DbiCascader.cxx` 

The :envvar:`DBCONF_PATH` is a colon delimited list of paths that are 
user (~) and $envvar OR ${envvar} expanded, some of the paths 
may not exist.  When there are repeated settings in more than one
file the last one wins.

In secure mode a single protected config file is required, the security 
comes with a high price in convenience

Definition at line 75 of file dbconf.py.

00075                                                                                                                                                                                                       : 
00076         """
00077 
00078         See also :ref:`dbi:running` section of the Offline User Manual 
00079 
00080         Interpolates the DB connection parameter patterns gleaned 
00081         from arguments, envvars or defaults (in that precedence order)
00082         into usable values using the context supplied by the 
00083         *sect* section of the ini format config file at *path*
00084 
00085         Optional keyword arguments:
00086      
00087                ================   =======================================================
00088                Keyword            Description
00089                ================   =======================================================
00090                   *sect*            section in config file 
00091                   *path*            colon delimited list of paths to config file
00092 
00093                   *user*            username 
00094                   *pswd*            password
00095                   *url*             connection url
00096                   *host*            db host 
00097                   *db*              db name
00098 
00099                   *fix*             triggers fixture loading into temporary 
00100                                     spawned cascade and specifies paths to fixture files
00101                                     for each member of the cascade (semi-colon delimited)  
00102                   *fixpass*         skip the DB cascade  dropping/creation that is 
00103                                     normally done
00104                                     as part of cascade spawning (used in DBWriter/tests) 
00105                   *restrict*        constrain the names of DB that can connect to 
00106                                     starting with a string, eg ``tmp_`` as a safeguard
00107                   *nodb*            used to connect without specifying the database
00108                                     this requires greater access privileges and is used
00109                                     to perform database dropping/creation
00110                ================   =======================================================
00111 
00112                                     
00113         Correspondingly named envvars can also be used:
00114 
00115                 .. envvar:: DBCONF 
00116                 .. envvar:: DBCONF_PATH  
00117 
00118                 .. envvar:: DBCONF_USER
00119                 .. envvar:: DBCONF_PWSD
00120                 .. envvar:: DBCONF_URL
00121                 .. envvar:: DBCONF_HOST
00122                 .. envvar:: DBCONF_DB
00123 
00124                 .. envvar:: DBCONF_FIX
00125                 .. envvar:: DBCONF_FIXPASS
00126                 .. envvar:: DBCONF_RESTRICT
00127 
00128         The :envvar:`DBCONF` existance also triggers the 
00129         :meth:`DybPython.dbconf.DBConf.Export` in :dybgaudi:`Database/DatabaseInterface/src/DbiCascader.cxx` 
00130 
00131         The :envvar:`DBCONF_PATH` is a colon delimited list of paths that are 
00132         user (~) and $envvar OR ${envvar} expanded, some of the paths 
00133         may not exist.  When there are repeated settings in more than one
00134         file the last one wins.
00135 
00136         In secure mode a single protected config file is required, the security 
00137         comes with a high price in convenience
00138 
00139         """
00140 
00141         self.secure = secure
00142         self.verbose = verbose
00143 
00144         esect = os.environ.get('DBCONF', None )
00145         if esect == "" or esect == None:
00146             esect = DBConf.defaults['sect']
00147 
00148         sect   = sect    or esect
00149         path   = path    or os.environ.get('DBCONF_PATH', DBConf.defaults['path'] ) 
00150 
00151         user   = user    or os.environ.get('DBCONF_USER', DBConf.defaults['user'] ) 
00152         pswd   = pswd    or os.environ.get('DBCONF_PSWD', DBConf.defaults['pswd'] ) 
00153         url    = url     or os.environ.get('DBCONF_URL',  DBConf.defaults['url'] ) 
00154         host   = host    or os.environ.get('DBCONF_HOST', DBConf.defaults['host'] ) 
00155         db     = db      or os.environ.get('DBCONF_DB'  , DBConf.defaults['db'] ) 
00156 
00157         fix    = fix     or os.environ.get('DBCONF_FIX' , DBConf.defaults['fix'] ) 
00158         fixpass = fixpass or os.environ.get('DBCONF_FIXPASS' , DBConf.defaults['fixpass'] ) 
00159         restrict = restrict or os.environ.get('DBCONF_RESTRICT' , DBConf.defaults['restrict'] ) 
00160 
00161 
00162         if self.secure:
00163             self._check_path( path )
00164         if not from_env:
00165             user,pswd,host,db,url = self.configure_cascade( sect, path ) 
00166  
00167         if restrict:
00168             dbns = db.split(";")
00169             dbok = filter( lambda _:_.startswith(restrict) , dbns )
00170             assert len(dbns) == len(dbok), "DBCONF_RESTRICTion on DB names violated : all DB names must be prefixed with \"%s\" DB names :\"%s\"  DB ok names: \"%s\"  " % ( restrict , dbns , dbok  )
00171 
00172 
00173         fsect = sect.split(":")[0]   ## first or only 
00174 
00175         self.sect = sect
00176         self.path = path
00177         self.user = user
00178         self.pswd = pswd
00179         self.url  = url
00180         self.host = host
00181         self.db   = db
00182         self.fix  = fix
00183         self.fixpass  = fixpass
00184         self.nodb = nodb
00185 

Member Function Documentation

def DybPython::dbconf::DBConf::Export (   cls,
  sect = None,
  extras 
)
Exports the environment settings into environment of python process 
this is invoked by the C++ *DbiCascader* ctor 

Definition at line 54 of file dbconf.py.

00055                                           :
00056         """
00057         Exports the environment settings into environment of python process 
00058         this is invoked by the C++ *DbiCascader* ctor 
00059         """
00060         if sect=="-":
00061             print "DBConf.Export noop on sect %s " % sect
00062             return
00063 
00064         cnf = DBConf( sect=sect )  
00065         if cnf.fix == None:
00066             cnf.export_to_env(**extras)
00067         else:
00068             from dbcas import DBCas
00069             cas = DBCas(cnf)
00070             tas = cas.spawn()
00071             cnf.export_to_env( supplier=tas )                
00072         return cnf
        #print dbc.dump_env()
def DybPython::dbconf::DBConf::mysqldb_parameters (   self,
  nodb = False 
)
Using the `nodb=True` option skips database name parameter, this is useful
when creating or dropping a database

Definition at line 186 of file dbconf.py.

00187                                             :
00188         """
00189         Using the `nodb=True` option skips database name parameter, this is useful
00190         when creating or dropping a database
00191         """
00192         #return dict(read_default_file=self.path, read_default_group=self.sect)
00193         d = dict(host=self.host % self, user=self.user % self, passwd=self.pswd % self ) 
00194         if not nodb:
00195             d['db']=self.db % self 
00196         if self.verbose:
00197             print "dbconf : connecting to %s " % dict(d, passwd="***" )
00198         return d
00199      

def DybPython::dbconf::DBConf::_check_path (   self,
  path 
) [private]
Check existance and permissions of path 

Definition at line 200 of file dbconf.py.

00201                                 :
00202         """
00203         Check existance and permissions of path 
00204         """ 
00205         assert os.path.exists( path ), "config path %s does not exist " % path
00206         from stat import S_IMODE, S_IRUSR, S_IWUSR
00207         s = os.stat(path)
00208         assert S_IMODE( s.st_mode ) == S_IRUSR | S_IWUSR , "incorrect permissions, config file must be protected with : chmod go-rw \"%s\" " %  path
00209 

def DybPython::dbconf::DBConf::configure_cascade (   self,
  sect,
  path 
)
Interpret the `sect` argument comprised of a either a single section name eg `offline_db`
or a colon delimited list of section names eg `tmp_offline_db:offline_db` 
to provide easy cascade configuration. A single section is of course a special case of a
cascade.  The first(or only) section in zeroth slot is treated specially with its config
parameters being propagated into `self`.

Caution any settings of `url`, `user`, `pswd`, `host`, `db` are overridden when
the `sect` argument contains a colon. 

Definition at line 210 of file dbconf.py.

00211                                             :
00212         """
00213         Interpret the `sect` argument comprised of a either a single section name eg `offline_db`
00214         or a colon delimited list of section names eg `tmp_offline_db:offline_db` 
00215         to provide easy cascade configuration. A single section is of course a special case of a
00216         cascade.  The first(or only) section in zeroth slot is treated specially with its config
00217         parameters being propagated into `self`.
00218 
00219         Caution any settings of `url`, `user`, `pswd`, `host`, `db` are overridden when
00220         the `sect` argument contains a colon. 
00221         """ 
00222         cfp, paths = DBConf.read_cfg( path )
00223         secs = cfp.sections()
00224         csect = sect.split(":")
00225         zsect = csect[0]          ## zeroth slot 
00226 
00227         if self.verbose:
00228             print "configure_cascade sect %s secs %r csect %r " % ( sect, secs, csect )
00229         cascade = {}
00230         for sect in csect:
00231             assert sect in secs  , "section %s is not one of these : %s configured in %s " % ( sect,  secs, paths ) 
00232             cascade[sect] = {}
00233             for k,v in cfp.items(sect):
00234                 cascade[sect][k] = v
00235  
00236         user = ";".join([ cascade[sect]['user'] for sect in csect ])
00237         pswd = ";".join([ cascade[sect]['password'] for sect in csect ])
00238         host = ";".join([ cascade[sect]['host'] for sect in csect ])
00239         db   = ";".join([ cascade[sect]['database']   for sect in csect ])
00240         url  = ";".join([ DBConf.defaults['url'] % cascade[sect] for sect in csect ])
00241 
00242         self.update( cascade[zsect] )  ##   zeroth slot into self
00243         self.cascade = cascade        ## for debugging only 
00244         return user, pswd, host, db, url
 
def DybPython::dbconf::DBConf::dump_env (   self,
  epfx = 'env_' 
)

Definition at line 245 of file dbconf.py.

00246                                    :
00247         e = {}
00248         for k,v in os.environ.items():
00249             if k.startswith(epfx.upper()):e.update({k:v} )   
00250         return e
00251 

def DybPython::dbconf::DBConf::smry (   self)

Definition at line 258 of file dbconf.py.

00259                   :
00260         print "urls  %s " % self.urls
00261         print "users %s " % self.users
00262         print "pswds %s " % self.pswds
00263         print "fixs  %s " % self.fixs

Construct :class:`DBConf` objects from environment : 

    :envvar:`ENV_TSQL_URL` 
    :envvar:`ENV_TSQL_USER` 
    :envvar:`ENV_TSQL_PSWD` 
    :envvar:`ENV_TSQL_FIX` 

Definition at line 264 of file dbconf.py.

00265                      :
00266         """
00267         Construct :class:`DBConf` objects from environment : 
00268 
00269             :envvar:`ENV_TSQL_URL` 
00270             :envvar:`ENV_TSQL_USER` 
00271             :envvar:`ENV_TSQL_PSWD` 
00272             :envvar:`ENV_TSQL_FIX` 
00273 
00274         """
00275         url  = os.environ.get( 'ENV_TSQL_URL', None )
00276         user = os.environ.get( 'ENV_TSQL_USER', None )
00277         pswd = os.environ.get( 'ENV_TSQL_PSWD', None )
00278         fix  = os.environ.get( 'ENV_TSQL_FIX' , None )
00279 
00280         #print "DBConf.from_env url %s user %s pwsd %s fix %s " % ( url, user, pswd, fix )
00281 
00282         assert url and user and pswd , "DBConf.from_env reconstruction requites the ENV_TSQL_* "
00283         cnf = DBConf(url=url, user=user, pswd=pswd, fix=fix, from_env=True)
        return cnf
def DybPython::dbconf::DBConf::export_ (   self,
  extras 
)
Exports the interpolated configuration into corresponding *DBI* envvars :

    :envvar:`ENV_TSQL_USER`
    :envvar:`ENV_TSQL_PSWD`
    :envvar:`ENV_TSQL_URL`
    :envvar:`ENV_TSQL_FIX`  (added to allow DBConf to survive thru the env-glass )
 
And *DatabaseSvc* envvars for access to non-DBI tables via DatabaseSvc :

    :envvar:`DYB_DB_USER`
    :envvar:`DYB_DB_PWSD`
    :envvar:`DYB_DB_URL`

Definition at line 287 of file dbconf.py.

00288                                :
00289         """
00290         Exports the interpolated configuration into corresponding *DBI* envvars :
00291 
00292             :envvar:`ENV_TSQL_USER`
00293             :envvar:`ENV_TSQL_PSWD`
00294             :envvar:`ENV_TSQL_URL`
00295             :envvar:`ENV_TSQL_FIX`  (added to allow DBConf to survive thru the env-glass )
00296          
00297         And *DatabaseSvc* envvars for access to non-DBI tables via DatabaseSvc :
00298 
00299             :envvar:`DYB_DB_USER`
00300             :envvar:`DYB_DB_PWSD`
00301             :envvar:`DYB_DB_URL`
00302 
00303         """ 
00304         supplier = extras.pop('supplier', None )
00305         if supplier:
00306             print "export_ supplier is %s " % supplier
00307         else:
00308             supplier = self
00309 
00310         self.export={}
00311         self.export['ENV_TSQL_URL'] =  supplier.url  % self 
00312         self.export['ENV_TSQL_USER'] = supplier.user % self 
00313         self.export['ENV_TSQL_PSWD'] = supplier.pswd % self 
00314         if supplier.fix:
00315             self.export['ENV_TSQL_FIX']  = supplier.fix % self 
00316 
00317         self.export['DYB_DB_HOST']   = supplier.host % self
00318         self.export['DYB_DB_NAME']   = supplier.db   % self
00319         self.export['DYB_DB_USER']   = supplier.user % self
00320         self.export['DYB_DB_PSWD']   = supplier.pswd % self
00321       
00322         for k,v in extras.items():
00323             self.export[k] = v % self 
00324 

def DybPython::dbconf::DBConf::read_cfg (   cls,
  path = None 
)
Classmethod to read config file(s) as specified by `path` argument or :envvar:`DBCONF_PATH` using 
:py:mod:`ConfigParser`  

Definition at line 325 of file dbconf.py.

00326                                    :
00327         """
00328         Classmethod to read config file(s) as specified by `path` argument or :envvar:`DBCONF_PATH` using 
00329         :py:mod:`ConfigParser`  
00330         """
00331         path = path or os.environ.get('DBCONF_PATH', DBConf.defaults['path'] ) 
00332         from ConfigParser import ConfigParser
00333         cfp = ConfigParser(DBConf.prime_parser())
00334         cfp.optionxform = str   ## avoid lowercasing keys, making the keys case sensitive
00335         paths = cfp.read( [os.path.expandvars(os.path.expanduser(p)) for p in path.split(":")] )   
        return cfp, paths
def DybPython::dbconf::DBConf::has_config (   cls,
  name_ = None 
)
Returns if the named config is available in any of the available DBCONF files 

For cascade configs (which comprise a colon delimited list of section names) all 
the config sections must be present.

As this module exposes this in its main, config sections can be tested on command line with::

    ./dbconf.py offline_db && echo y || echo n 
    ./dbconf.py offline_dbx && echo y || echo n  
    ./dbconf.py tmp_offline_db:offline_db && echo y || echo n
    ./dbconf.py tmp_offline_dbx:offline_db && echo y || echo n

Definition at line 339 of file dbconf.py.

00340                                       :
00341         """
00342         Returns if the named config is available in any of the available DBCONF files 
00343 
00344         For cascade configs (which comprise a colon delimited list of section names) all 
00345         the config sections must be present.
00346 
00347         As this module exposes this in its main, config sections can be tested on command line with::
00348 
00349             ./dbconf.py offline_db && echo y || echo n 
00350             ./dbconf.py offline_dbx && echo y || echo n  
00351             ./dbconf.py tmp_offline_db:offline_db && echo y || echo n
00352             ./dbconf.py tmp_offline_dbx:offline_db && echo y || echo n
00353 
00354         """ 
00355         if not name_:
00356             name_ = os.environ.get('DBCONF',None)
00357         assert name_, "has_config requires an argument if DBCONF envvar is not defined "  
00358         cfp, paths = DBConf.read_cfg()
00359         sects = cfp.sections()
00360         if ":" in name_:
00361             names = name_.split(":")
00362             ok_names = filter(lambda _:_ in sects, names )
00363             return len(names) == len(ok_names)
00364         else:
            return name_ in sects           
Prime parser with "today" to allow expansion of ``%(today)s`` in ``~/.my.cnf``
allowing connection to a daily recovered database named after todays date

Definition at line 367 of file dbconf.py.

00368                            :
00369         """
00370         Prime parser with "today" to allow expansion of ``%(today)s`` in ``~/.my.cnf``
00371         allowing connection to a daily recovered database named after todays date
00372         """
00373         from datetime import datetime
        return dict(today=datetime.now().strftime("%Y%m%d"))
def DybPython::dbconf::DBConf::export_to_env (   self,
  extras 
)

Definition at line 377 of file dbconf.py.

00378                                      :
00379         self.export_(**extras)
00380         if self.verbose:
00381             print "dbconf:export_to_env from %s section %s " % ( self.path, self.sect ) 
00382         os.environ.update(self.export) 
00383         if self.verbose:
00384             print " ==> %s " % dict(self.export, ENV_TSQL_PSWD='***', DYB_DB_PSWD='***' ) 


Member Data Documentation

Initial value:
{ 
                 'path':"$SITEROOT/../.my.cnf:~/.my.cnf", 
                 'sect':"offline_db",
                 'host':"%(host)s", 
                 'user':"%(user)s", 
                   'db':"%(database)s", 
                 'pswd':"%(password)s",
                  'url':"mysql://%(host)s/%(database)s", 
                  'fix':None,
                  'fixpass':None,
                  'restrict':None,
               }

Definition at line 41 of file dbconf.py.

tuple DybPython::dbconf::DBConf::Export = classmethod( Export ) [static]

Definition at line 73 of file dbconf.py.

tuple DybPython::dbconf::DBConf::from_env = classmethod(from_env) [static]

Definition at line 284 of file dbconf.py.

tuple DybPython::dbconf::DBConf::read_cfg = classmethod( read_cfg ) [static]

Definition at line 336 of file dbconf.py.

tuple DybPython::dbconf::DBConf::has_config = classmethod( has_config ) [static]

Definition at line 365 of file dbconf.py.

tuple DybPython::dbconf::DBConf::prime_parser = classmethod( prime_parser ) [static]

Definition at line 374 of file dbconf.py.

string DybPython::dbconf::DBConf::__str__ = "\n" [static, private]

Definition at line 385 of file dbconf.py.

Definition at line 138 of file dbconf.py.

Definition at line 138 of file dbconf.py.

first or only

Definition at line 139 of file dbconf.py.

Definition at line 139 of file dbconf.py.

Definition at line 139 of file dbconf.py.

Definition at line 139 of file dbconf.py.

Definition at line 139 of file dbconf.py.

Definition at line 139 of file dbconf.py.

Definition at line 139 of file dbconf.py.

Definition at line 139 of file dbconf.py.

Definition at line 139 of file dbconf.py.

Definition at line 139 of file dbconf.py.

zeroth slot

zeroth slot into self

Definition at line 221 of file dbconf.py.

Definition at line 301 of file dbconf.py.


Property Documentation

DybPython::dbconf::DBConf::urls = property( lambda self:(self.url % self).split(";") ) [static]

Definition at line 252 of file dbconf.py.

DybPython::dbconf::DBConf::users = property( lambda self:(self.user % self).split(";") ) [static]

Definition at line 253 of file dbconf.py.

DybPython::dbconf::DBConf::pswds = property( lambda self:(self.pswd % self).split(";") ) [static]

Definition at line 254 of file dbconf.py.

DybPython::dbconf::DBConf::fixs = property( lambda self:(self.fix % self).split(";") ) [static]

Definition at line 255 of file dbconf.py.


The documentation for this class was generated from the following file:
| Classes | Job Modules | Data Objects | Services | Algorithms | Tools | Packages | Directories | Tracs |

Generated on Fri May 16 2014 09:55:40 for DybPython by doxygen 1.7.4