/search.css" rel="stylesheet" type="text/css"/> /search.js">
| Classes | Job Modules | Data Objects | Services | Algorithms | Tools | Packages | Directories | Tracs |

In This Package:

Public Member Functions | Public Attributes
ctxmap::DbiCtxCopy Class Reference

List of all members.

Public Member Functions

def __init__
def snapshot
def ssmap
def check
def __call__

Public Attributes

 db
 kls
 tn

Detailed Description

Definition at line 139 of file ctxmap.py.


Constructor & Destructor Documentation

def ctxmap::DbiCtxCopy::__init__ (   self,
  db,
  kls 
)
:param db: `DB` instance
:param kls: DybDbi class

Definition at line 140 of file ctxmap.py.

00141                                 :
00142         """
00143         :param db: `DB` instance
00144         :param kls: DybDbi class
00145         """
00146         dict.__init__(self)
00147         self.db = db
00148         self.kls = kls
00149         self.tn = kls.__name__[1:]


Member Function Documentation

def ctxmap::DbiCtxCopy::snapshot (   self,
  path 
)
Obtain table counts and ranges from `group by` query and persist as pickled dict-of-dict 
tuple keyed with (SITEMASK,SUBSITE)  

:param path: in which to persist the dict-of-dict
:return: `VGroup` instance

Definition at line 150 of file ctxmap.py.

00151                                :
00152         """
00153         Obtain table counts and ranges from `group by` query and persist as pickled dict-of-dict 
00154         tuple keyed with (SITEMASK,SUBSITE)  
00155 
00156         :param path: in which to persist the dict-of-dict
00157         :return: `VGroup` instance
00158         """
00159         vg = VGroup(xf="SITEMASK SUBSITE", qf="SEQNO", tf="TIMESTART TIMEEND INSERTDATE" )
00160         vg(self.db,self.tn) 
00161         vg.persist(path)
00162         return vg

def ctxmap::DbiCtxCopy::ssmap (   self,
  vmp 
)
Invoke the mapping call adding as many 
new entries as there are pre-existing ones  

:param vmp: `VMapping` instance

Definition at line 163 of file ctxmap.py.

00164                         :
00165         """
00166         Invoke the mapping call adding as many 
00167         new entries as there are pre-existing ones  
00168 
00169         :param vmp: `VMapping` instance
00170         """
00171         self.clear()                 # scrub dict holding the seqno mappings used for checking 
00172         expect = vmp.expect(self.db)
00173         log.info("ssmap for VMapping %s expects %s copied vrec (with payload) " % ( vmp, expect ))
00174         nwr = self(vmp['sqlcontext'], tgt_sitemask=vmp['tgt_sitemask'] , tgt_subsite=vmp['tgt_subsite'] )
00175         assert nwr == expect, (nwr, expect )
00176         self.check(nwr)

def ctxmap::DbiCtxCopy::check (   self,
  nwr 
)
:param nwr: expected number of writes

Check the mapping between SEQNO read and SEQNO written conforms
to expectations, expect contiguous block of writes 

Definition at line 177 of file ctxmap.py.

00178                          :
00179         """
00180         :param nwr: expected number of writes
00181 
00182         Check the mapping between SEQNO read and SEQNO written conforms
00183         to expectations, expect contiguous block of writes 
00184 
00185         """
00186         assert len(self) == nwr, (len(self),nwr)
00187 
00188         keys = sorted(self.keys())
00189         vals = sorted(self.values())
00190         rseq = min(keys),max(keys) 
00191         wseq = min(vals),max(vals) 
00192 
00193         # assert rseq[1] - rseq[0] + 1 == nwr, (rseq, nwr )    nope not contiguous on the source side,  AssertionError: ((47614L, 84331L), 4584)
00194         assert wseq[1] - wseq[0] + 1 == nwr, (wseq, nwr ) 
00195         assert self[rseq[0]] == wseq[0]
00196         assert self[rseq[1]] == wseq[1]
00197 
00198         for rq,wq in zip(keys,vals):
00199             assert self[rq] == wq 
00200 

def ctxmap::DbiCtxCopy::__call__ (   self,
  sqlcontext = "1=1",
  tgt_sitemask = None,
  tgt_subsite = None 
)
:param sqlcontext: constituency of validity records to be duplicated together with their payloads 
:param tgt_sitemask:
:param tgt_subsite:
:return: number of validity records duplicated

Definition at line 201 of file ctxmap.py.

00202                                                                               :
00203         """
00204         :param sqlcontext: constituency of validity records to be duplicated together with their payloads 
00205         :param tgt_sitemask:
00206         :param tgt_subsite:
00207         :return: number of validity records duplicated
00208         """
00209         src_task, src_subsite =  -1, -1           ## wideopen DBI querying restricted by sqlcontext only 
00210         vrb = self.kls.GetTableProxy().MakeValidityRecBuilder( sqlcontext, src_subsite, src_task )
00211         nvr = vrb.GetNumValidityRec()
00212         gap = vrb.GetValidityRec(0)
00213         assert gap.IsGap() 
00214         log.info("nvr %s " % nvr )
00215 
00216         nwr = 0
00217         for ivr in reversed(range(1,nvr)):   ## reverse for closer SEQNO correspondence 
00218             vrec = vrb.GetValidityRec(ivr)
00219 
00220             # wideopen reader for all payloads within the vrec (aka SEQNO)
00221             rpt = self.kls.Rpt().Clone()
00222             rpt.ctx( validityrec=vrec )  ## vrec reader does " select * from T where SEQNO= 113 "   etc...
00223             n = len(rpt)
00224             if ivr % 100 == 0:
00225                 log.info("ivr %s vrec seqno %s n %s " % (ivr, vrec.seqno, n ))
00226             log.debug("ivr %s vrec seqno %s n %s " % (ivr, vrec.seqno, n ))
00227    
00228             cr = ContextRange( vrec.contextrange.sitemask , vrec.contextrange.simmask , vrec.contextrange.timestart, vrec.contextrange.timeend )
00229             if tgt_sitemask:
00230                 cr.sitemask = tgt_sitemask
00231             new_subsite = tgt_subsite if tgt_subsite else vrec.subsite
00232 
00233             # fresh writer for each vrec 
00234             wrt = self.kls.Wrt().Clone() 
00235             wrt.ctx( contextrange=cr, subsite=new_subsite, task=vrec.task, versiondate=vrec.versiondate, tablename=self.tn ) 
00236 
00237             # write all source payloads to target 
00238             for row in rpt:
00239                 wrt.Write(row)    
00240             wseqno = wrt.Close()
00241             nwr += 1
00242             assert wseqno, "writer failed to close %s %s " % (ivr, vrec ) 
00243             self[vrec.seqno] = wseqno 
00244             pass
00245         log.info("from %s attempted to propagate %s " % (nvr-1, nwr) )   ## slot 0 is Gap 
00246         assert nvr - 1 == nwr , (nvr - 1,nwr)
00247         return nwr
00248 


Member Data Documentation

Definition at line 143 of file ctxmap.py.

Definition at line 143 of file ctxmap.py.

Definition at line 143 of file ctxmap.py.


The documentation for this class was generated from the following file:
| Classes | Job Modules | Data Objects | Services | Algorithms | Tools | Packages | Directories | Tracs |

Generated on Fri May 16 2014 09:57:24 for DybDbi by doxygen 1.7.4