/search.css" rel="stylesheet" type="text/css"/> /search.js">
| Classes | Job Modules | Data Objects | Services | Algorithms | Tools | Packages | Directories | Tracs |

In This Package:

Public Member Functions | Public Attributes | Static Public Attributes | Properties | Private Member Functions | Private Attributes
DybDbi::csvrw::CSV Class Reference

List of all members.

Public Member Functions

def __init__
def read
def is_valid
def blanks
def smry
def __repr__
def write

Public Attributes

 kwargs
 stat
 src

Static Public Attributes

tuple defaults = dict( delimiter="\t", prefix="#Table", descmarker="#[]", synth="_srcline" , fields=[] )

Properties

 path = property(lambda self:os.path.expanduser(os.path.expandvars(self._path)))
 delimiter = property(lambda self:self.kwargs.get('delimiter',self.defaults.get('delimiter',None)))
 fieldnames = property(_fieldnames, doc=_fieldnames.__doc__)
 descline = property(lambda self:"#" + self.delimiter.join(self.fieldnames) )

Private Member Functions

def _fieldnames

Private Attributes

 _path

Detailed Description

Reader/writer for .csv files. The contents are stored as a list of dicts.

:param delimiter: csv field divider
:param prefix:  string start of comment lines to be ignored, default #Table
:param descmarker: strings used to identify the field description line 
:param synth: when defined, add extra field with this name to hold the csv source line number 
:param fields: impose fieldnames externally, useful for handling broken csv which cannot be fixed immediately 

Read usage example::

    src = CSV("$DBWRITERROOT/share/DYB_MC_AD1.txt", delimiter="\\t" )
    src.read()
    for d in src:
        print d

    len(src)
    src[0]
    src[-1]
    src.fieldnames


On reading an invalid CSV an exception, with error report, is raised::

    src = CSV("$DBWRITERROOT/share/DYB_SAB_AD1.txt", delimiter="\\t" )
    src.read()


Handling of common csv incorrectnesses is made:

#. description line fixed up to conform to the delimiter
#. description line extraneous characters removed (other than fieldnames and delimiters)
#. removes comments 
 
Write usage example, field names are obtained from the dict keys::

     out = CSV("/tmp/demo.csv", delimiter="\\t" )
     for d in list_of_dict_datasource:
         out.append(d)
     out.write()

Definition at line 3 of file csvrw.py.


Constructor & Destructor Documentation

def DybDbi::csvrw::CSV::__init__ (   self,
  path,
  kwargs 
)
       

Definition at line 48 of file csvrw.py.

00049                                        :
00050         """
00051        """ 
00052         self._path = path
00053         self.kwargs = kwargs
00054         self.stat = dict( hasblank=0 )


Member Function Documentation

def DybDbi::csvrw::CSV::_fieldnames (   self) [private]
If fieldnames keyword argument is supplied return that otherwise
return the names of the keys in the first contained dict.  In order to
control the order of fields, the argument has to be specified.

Definition at line 55 of file csvrw.py.

00056                          :
00057         """
00058         If fieldnames keyword argument is supplied return that otherwise
00059         return the names of the keys in the first contained dict.  In order to
00060         control the order of fields, the argument has to be specified.
00061         """
00062         fns = self.kwargs.get('fieldnames',None)
00063         if fns:
00064             return fns
00065         if len(self)>0:
00066             return self[0].keys()     
00067         return None

def DybDbi::csvrw::CSV::read (   self)

Definition at line 73 of file csvrw.py.

00074                   :
00075         kwargs = self.kwargs
00076         delimiter = kwargs.pop('delimiter', self.defaults['delimiter'])
00077         prefix =    kwargs.pop('prefix', self.defaults['prefix'])
00078         descmarker = kwargs.pop('descmarker',self.defaults['descmarker'])
00079         synth =     kwargs.pop('synth',self.defaults['synth'])
00080         fields =    kwargs.pop('fields',self.defaults['fields'])
00081 
00082         src = Source(open(self.path,"r"), delimiter=delimiter, prefix=prefix, descmarker=descmarker, synth=synth, fields=fields )
00083 
00084         for i,r in enumerate(csv.DictReader(src,delimiter=delimiter)):
00085             row = Entry(r)
00086             hasblank = row.hasblank
00087             if hasblank:
00088                 self.stat['hasblank'] += 1 
00089             row.update( _hasblank=hasblank )
00090             self.append(row)
00091         self.src = src
00092 
00093         if not self.is_valid():
00094             self.smry()
00095             raise Exception("CSV.read INVALID .csv :  %r " % self )  
00096 

def DybDbi::csvrw::CSV::is_valid (   self)

Definition at line 97 of file csvrw.py.

00098                       :
00099         return len(self.blanks()) == 0 

def DybDbi::csvrw::CSV::blanks (   self)

Definition at line 100 of file csvrw.py.

00101                     :
00102         return filter( lambda _:_['_hasblank'] == True, self)

def DybDbi::csvrw::CSV::smry (   self)

Definition at line 103 of file csvrw.py.

00104                   :
00105         print "source csv...", self.src
00106         print "CSV ... ", self
00107         print "irregularities in csv..."
00108         for blk in self.blanks():
00109             print blk
00110         for blk in self.blanks():
00111             print self.src[int(blk['_srcline'])]
   
def DybDbi::csvrw::CSV::__repr__ (   self)

Definition at line 112 of file csvrw.py.

00113                       :
00114         return "CSV %s %s rows %s stat %r  " % ( self._path, ["NOT VALID","valid"][self.is_valid()], len(self), self.stat )

def DybDbi::csvrw::CSV::write (   self)

Definition at line 115 of file csvrw.py.

00116                    :
00117         """
00118         """
00119         out = open(self.path,"w")
00120         out.write("#Table CSV.write \n" )  
00121         out.write( self.descline + "\n")
00122         writer = csv.DictWriter( out , self.fieldnames , delimiter=self.delimiter )
00123         for d in self:
00124             writer.writerow(d)
00125         out.close()
00126 
00127 


Member Data Documentation

tuple DybDbi::csvrw::CSV::defaults = dict( delimiter="\t", prefix="#Table", descmarker="#[]", synth="_srcline" , fields=[] ) [static]

Definition at line 46 of file csvrw.py.

Definition at line 49 of file csvrw.py.

Definition at line 49 of file csvrw.py.

Definition at line 49 of file csvrw.py.

Definition at line 73 of file csvrw.py.


Property Documentation

DybDbi::csvrw::CSV::path = property(lambda self:os.path.expanduser(os.path.expandvars(self._path))) [static]

Definition at line 68 of file csvrw.py.

DybDbi::csvrw::CSV::delimiter = property(lambda self:self.kwargs.get('delimiter',self.defaults.get('delimiter',None))) [static]

Definition at line 69 of file csvrw.py.

DybDbi::csvrw::CSV::fieldnames = property(_fieldnames, doc=_fieldnames.__doc__) [static]

Definition at line 70 of file csvrw.py.

DybDbi::csvrw::CSV::descline = property(lambda self:"#" + self.delimiter.join(self.fieldnames) ) [static]

Definition at line 71 of file csvrw.py.


The documentation for this class was generated from the following file:
| Classes | Job Modules | Data Objects | Services | Algorithms | Tools | Packages | Directories | Tracs |

Generated on Fri May 16 2014 09:57:24 for DybDbi by doxygen 1.7.4