/search.css" rel="stylesheet" type="text/css"/> /search.js">
| Classes | Job Modules | Data Objects | Services | Algorithms | Tools | Packages | Directories | Tracs |

In This Package:

Public Member Functions | Public Attributes
Scraper::daq::fileinfo::FileInfo Class Reference
Inheritance diagram for Scraper::daq::fileinfo::FileInfo:
Inheritance graph
[legend]
Collaboration diagram for Scraper::daq::fileinfo::FileInfo:
Collaboration graph
[legend]

List of all members.

Public Member Functions

def __init__
def __repr__
def last
def after

Public Attributes

 daqf
 offf

Detailed Description

Definition at line 109 of file fileinfo.py.


Constructor & Destructor Documentation

def Scraper::daq::fileinfo::FileInfo::__init__ (   self,
  source = "tmp_daqdb",
  target = "tmp_offline_db",
  cfg = {} 
)

Definition at line 110 of file fileinfo.py.

00110                                                                          {} ):
00111         BaseInfo.__init__(self, source, target, cfg) 
00112         self.daqf = DaqFiles( self.source, cfg  )
00113         self.offf = OffFiles( self.target, cfg  )
00114 

Member Function Documentation

def Scraper::daq::fileinfo::FileInfo::__repr__ (   self)

Definition at line 115 of file fileinfo.py.

00116                       :
00117         return BaseInfo.__repr__(self)

def Scraper::daq::fileinfo::FileInfo::last (   self)

Definition at line 118 of file fileinfo.py.

00119                   :
00120         return self.offf.last()

def Scraper::daq::fileinfo::FileInfo::after (   self,
  startlocal,
  offset_seconds = None 
)
Queries daq table SFO_TZ_FILE for closed files, that were opened after cutoff time `startlocal` 
and that are not already in offline table DaqRawDataFileInfo

Former ``N+1`` nested approach ``get_newFileList``:

* iterated over DAQ file entries, doing offline queries at each iteration to check for preexisting file names 

New flat ``3`` query approach:

#. runs corresponding to new DAQ files
#. offline files names corresponding to those runs
#. DAQ files with names not already in offline 

Could be done flat ``2``, but that would entail IN querying with all filenames, which is not sustainable.

Definition at line 121 of file fileinfo.py.

00122                                                       :
00123         """
00124         Queries daq table SFO_TZ_FILE for closed files, that were opened after cutoff time `startlocal` 
00125         and that are not already in offline table DaqRawDataFileInfo
00126 
00127         Former ``N+1`` nested approach ``get_newFileList``:
00128 
00129         * iterated over DAQ file entries, doing offline queries at each iteration to check for preexisting file names 
00130 
00131         New flat ``3`` query approach:
00132 
00133         #. runs corresponding to new DAQ files
00134         #. offline files names corresponding to those runs
00135         #. DAQ files with names not already in offline 
00136 
00137         Could be done flat ``2``, but that would entail IN querying with all filenames, which is not sustainable.
00138 
00139         """
00140         if not offset_seconds:
00141             offset_seconds = self.cfg.get('file_seconds_offset', -60 )
00142 
00143         startlocal = startlocal + timedelta(seconds=offset_seconds)
00144 
00145         daqruns = self.daqf.runs( startlocal )
00146         log.info("qfter %s yields %s daqruns " % ( startlocal, len(daqruns) )) 
00147 
00148         daqfiles = []
00149         if len(daqruns) == 0:
00150             log.info("no new closed runs after %s " % startlocal )
00151         else:
00152             offnames = self.offf.offnames(daqruns) 
00153             daqfiles = self.daqf.newfiles( startlocal, offnames )
00154         return daqfiles
00155 
00156 
00157 
00158 


Member Data Documentation

Definition at line 110 of file fileinfo.py.

Definition at line 110 of file fileinfo.py.


The documentation for this class was generated from the following file:
| Classes | Job Modules | Data Objects | Services | Algorithms | Tools | Packages | Directories | Tracs |

Generated on Fri May 16 2014 09:50:03 for Scraper by doxygen 1.7.4