/search.css" rel="stylesheet" type="text/css"/> /search.js">
| Classes | Job Modules | Data Objects | Services | Algorithms | Tools | Packages | Directories | Tracs |

In This Package:

Public Member Functions | Public Attributes | Private Member Functions
Scraper::base::faker::Faker Class Reference

List of all members.

Public Member Functions

def __init__
def __repr__
def __call__

Public Attributes

 cfg
 sleep
 interval
 timestart
 maxiter
 profile

Private Member Functions

def _insertfake

Detailed Description

create fake source instances and insert them 

Definition at line 7 of file faker.py.


Constructor & Destructor Documentation

def Scraper::base::faker::Faker::__init__ (   self,
  srcs,
  cfg 
)

Definition at line 11 of file faker.py.

00012                                   :
00013         self.cfg = cfg
00014         log.info("faking srcs %s from %s  " % ( srcs, srcs[0].db ) )
00015         for src in srcs:
00016             self.append(src)
00017         #
00018         self.sleep = cfg.pop('sleep') 
00019         self.interval = cfg.pop('interval') 
00020         self.timestart = cfg.pop('faker_timestart')
00021 
00022         self.maxiter = cfg.pop('maxiter')
00023         self.profile = cfg.pop('profile', None)  
00024 
00025         if len(cfg)>0:
00026             log.warn("%s unpopped cfg %r " % (self.__class__.__name__, cfg ) )
00027 


Member Function Documentation

def Scraper::base::faker::Faker::_insertfake (   self,
  index 
) [private]
:param index:  None or zero based fake index, typically the iteration loop counter 

An index of `None` causes and SA source query to be performed to determine the 
`id` of the last source entry for each source. 

Definition at line 28 of file faker.py.

00029                                  :
00030         """
00031         :param index:  None or zero based fake index, typically the iteration loop counter 
00032 
00033         An index of `None` causes and SA source query to be performed to determine the 
00034         `id` of the last source entry for each source. 
00035         """
00036         for src in self:
00037             
00038             inst = src()
00039             if index == None:        
00040                 last = src.last()   
00041                 id = 1000 if last == None else last.id
00042                 id += 1
00043             else:
00044                 id = index  
00045             dt = self.timestart + (index*self.interval)
00046 
00047             #
00048             # set defaults here, users can override if needed 
00049             # NOPE :. not so simple as inst could be a join with multiple id/date_time to set 
00050             # so for now leave it to specialized code
00051             #
00052             # setattr( inst , 'id' , id ) 
00053             # setattr( inst , 'date_time' , dt ) 
00054             #
00055 
00056             self.fake( inst , id , dt )
00057 
00058             log.info("IF %-3d %s %s %r " % (id, inst.date_time.strftime("%Y-%m-%d %H:%M:%S"), src.__name__, filter(lambda _:type(_) != datetime, inst.asdict.values() )))
00059             db = src.db
00060             db.add(inst)   
00061             db.commit()

def Scraper::base::faker::Faker::__repr__ (   self)

Definition at line 62 of file faker.py.

00063                       :
00064         return "%s %s " % ( self.__class__.__name__, self.sleep )

def Scraper::base::faker::Faker::__call__ (   self)

Definition at line 65 of file faker.py.

00065                       : 
00066 
00067         i = 0 
00068         while i<self.maxiter or self.maxiter==0:
00069             self._insertfake(i)
00070             time.sleep(self.sleep.seconds)
00071             i += 1
00072 
00073 
00074 

Member Data Documentation

Definition at line 11 of file faker.py.

Definition at line 11 of file faker.py.

Definition at line 11 of file faker.py.

Definition at line 11 of file faker.py.

Definition at line 11 of file faker.py.

Definition at line 11 of file faker.py.


The documentation for this class was generated from the following file:
| Classes | Job Modules | Data Objects | Services | Algorithms | Tools | Packages | Directories | Tracs |

Generated on Fri May 16 2014 09:50:03 for Scraper by doxygen 1.7.4