/search.css" rel="stylesheet" type="text/css"/> /search.js">
| Classes | Job Modules | Data Objects | Services | Algorithms | Tools | Packages | Directories | Tracs |

In This Package:

Public Member Functions | Public Attributes | Static Public Attributes
Scraper::base::utils::Utils Class Reference

List of all members.

Public Member Functions

def __init__
def drop_dbi_tables
def setup_faker_source
def setup_faker_source_

Public Attributes

 cfg

Static Public Attributes

tuple setup_faker_source_ = classmethod( setup_faker_source_ )

Detailed Description

Definition at line 5 of file utils.py.


Constructor & Destructor Documentation

def Scraper::base::utils::Utils::__init__ (   self,
  cfg 
)

Definition at line 6 of file utils.py.

00007                             :
00008         self.cfg = cfg


Member Function Documentation

def Scraper::base::utils::Utils::drop_dbi_tables (   self)

Definition at line 9 of file utils.py.

00010                              :
00011         cfg = self.cfg
00012         if not cfg['DROP_TARGET_TABLES']:
00013             return 
00014         target = cfg['target'] 
00015 
00016         assert target.startswith('tmp_'), "accident avoidance : dropable target must have sectnames beginning tmp_  "
00017         assert cfg['kls'][0] == 'G' , "unexpected kls %s " % cfg['kls']
00018         table = cfg['kls'][1:]
00019         assert table.startswith('DcsAdLid') , "unexpected table %s " % table
00020 
00021         from DybPython import DBConf
00022         tdbc = DBConf(target)
00023         tdbc['table'] = table
00024         assert tdbc['database'][0:4] == 'tmp_' 
00025         log.info("drop_dbi_tables %s " % target )
00026         ddt = DropDbiTables( tdbc )
00027         ddt()
 
def Scraper::base::utils::Utils::setup_faker_source (   self)

Definition at line 28 of file utils.py.

00029                                   :
00030         self.setup_faker_source_( self.cfg )

Drops, creates and loads fake source DB 

As this needs to be done before regime instanciation use 
classmethod and invoke from main

For cfg with `dropsrc` option DROP/CREATE/LOAD 
source DB from a mysqldump file at eg `~/fake_dcs.sql` where the 
basename corresponds to the source DBCONF sectionname, here `fake_dcs`

Without this get integrity errors from non-unique PKs if do not start 
from scratch

Definition at line 31 of file utils.py.

00032                                         :
00033         """
00034         Drops, creates and loads fake source DB 
00035 
00036         As this needs to be done before regime instanciation use 
00037         classmethod and invoke from main
00038 
00039         For cfg with `dropsrc` option DROP/CREATE/LOAD 
00040         source DB from a mysqldump file at eg `~/fake_dcs.sql` where the 
00041         basename corresponds to the source DBCONF sectionname, here `fake_dcs`
00042 
00043         Without this get integrity errors from non-unique PKs if do not start 
00044         from scratch
00045         """
00046         log.info('setup_faker_source')
00047         if not cfg['faker_dropsrc']:
00048             return 
00049         source = cfg['source'] 
00050         assert source.startswith('fake_'), "accident avoidance : dropable fake sources must have sectnames beginning fake_  "
00051         from DybPython import DBConf
00052         sdbc = DBConf(source)
00053         assert sdbc['database'] == 'fake_dcs' 
00054         log.info("setup_faker_source drop/create/load of source %s " % source ) 
00055         dcl = DropCreateLoad( sdbc, dumppath="~/%s.sql" % source )
00056         dcl()
        cfg['_setup_faker_source'] = True

Member Data Documentation

Definition at line 57 of file utils.py.

Definition at line 6 of file utils.py.


The documentation for this class was generated from the following file:
| Classes | Job Modules | Data Objects | Services | Algorithms | Tools | Packages | Directories | Tracs |

Generated on Fri May 16 2014 09:50:03 for Scraper by doxygen 1.7.4