/search.css" rel="stylesheet" type="text/css"/> /search.js">
| Classes | Job Modules | Data Objects | Services | Algorithms | Tools | Packages | Directories | Tracs |

In This Package:

Public Member Functions | Public Attributes
DybDbi::csvrw::Source Class Reference

List of all members.

Public Member Functions

def __init__
def is_descline
def clean
def descline
def next
def __iter__
def __repr__

Public Attributes

 f
 prefix
 delimiter
 descmarker
 fields
 synth
 stat
 cols
 skip lines beginning with the prefix

Detailed Description

Behaves like a file and holds the original text of the CSV. 
Applies some fixes to make readable as CSV:

#. removes lines that begin with the ``prefix`` argument, default ``#Table``
#. determines the description line by looking for all characters of the ``descmarker`` argument, default ``#[]``
#. normalize the description line to conform to the delimiter

Normally used internally via CSV, but can be useful to debug broken .csv files interactively:

.. code-block:: ipython

   In [2]: import os

   In [3]: from DybDbi import Source

   In [4]: src=Source(open(os.path.expandvars("$DATASVCROOT/share/feeCableMap_MDC09a.txt")), delimiter=" ", descmarker="#" )

   In [5]: for _ in src:print _   ## have to interate to populate 

   In [7]: src
   Out[7]: 
   Source stat:{'descline': 1, 'prefix': 0, 'total': 1585, 'payload': 1584} 
    cols:['srcline', 'ChannelID', 'Description', 'ElecHardwareId', 'Description', 'SensorID', 'Description', 'SensorHardwareID', 'Description']  

   In [8]: src.descmarker
   Out[8]: '#'

   In [9]: src.descline( src[0] )   ## debugging the field extraction from the description line
   Out[9]: ' ChannelID Description ElecHardwareId Description SensorID Description SensorHardwareID Description'

Note the severely invalid .csv (4 fields with the same name) workaround until the .csv can be fixed is to 
externally impose the fields:

   .. code-block:: python 

   fields = 'ChannelID Description0 ElecHardwareId Description1 SensorID Description2 SensorHardwareID Description3'.split()
   src=Source(open(os.path.expandvars("$DATASVCROOT/share/feeCableMap_MDC09a.txt")), delimiter=" ", descmarker="#", fields=fields ) 

 

Definition at line 133 of file csvrw.py.


Constructor & Destructor Documentation

def DybDbi::csvrw::Source::__init__ (   self,
  f,
  delimiter = "\t",
  prefix = "#Table",
  descmarker = "#[]",
  synth = "srcline",
  fields = [] 
)
:param delimiter: csv field divider
:param prefix:  string start of lines to be ignored
:param descmarker: strings used to identify the field description line 
:param synth: when defined, add extra field with this name to hold the csv source line number 
:param fields: when defined overrides the content of the descline

Definition at line 175 of file csvrw.py.

00176                                                                                                          :
00177         """
00178         :param delimiter: csv field divider
00179         :param prefix:  string start of lines to be ignored
00180         :param descmarker: strings used to identify the field description line 
00181         :param synth: when defined, add extra field with this name to hold the csv source line number 
00182         :param fields: when defined overrides the content of the descline
00183         """
00184         self.f = f
00185         self.prefix = prefix
00186         self.delimiter = delimiter
00187         self.descmarker = descmarker
00188         self.fields = fields
00189         self.synth = synth
00190 
00191         self.stat = dict(total=0,prefix=0, payload=0, descline=0)
00192         self.cols = None


Member Function Documentation

def DybDbi::csvrw::Source::is_descline (   self,
  line 
)
Checks if line contains all of the description markers

Definition at line 193 of file csvrw.py.

00194                                :
00195         """
00196         Checks if line contains all of the description markers
00197         """
00198         return len(filter(lambda _:_ in line,self.descmarker)) == len(self.descmarker)

def DybDbi::csvrw::Source::clean (   self,
  line 
)
shrink multiple spaces to a single space, and strip head and tail whitespace

Definition at line 199 of file csvrw.py.

00200                          :
00201         """ 
00202         shrink multiple spaces to a single space, and strip head and tail whitespace
00203         """
00204         line = re.sub(' +',' ',line) 
00205         return line.lstrip().rstrip()

def DybDbi::csvrw::Source::descline (   self,
  line 
)
Remove the descmarker characters from the description line,

Definition at line 206 of file csvrw.py.

00207                             :
00208         """
00209         Remove the descmarker characters from the description line,
00210         """
00211         if len(self.fields) > 0:
00212             line = self.delimiter.join( self.fields )
00213         else:
00214             for c in self.descmarker:
00215                 line = line.replace(c,"")
00216         return self.clean(line)

def DybDbi::csvrw::Source::next (   self)
On iterating though this "synthetic" file fixes and additions are made to 
render the "physicist-csv" as real csv  

Definition at line 217 of file csvrw.py.

00218                   :
00219         """
00220         On iterating though this "synthetic" file fixes and additions are made to 
00221         render the "physicist-csv" as real csv  
00222         """
00223         line = self.f.next()
00224         self.append(line.rstrip())
00225         self.stat['total'] += 1
00226         ## skip lines beginning with the prefix
00227         while line.startswith(self.prefix):  
00228             self.stat['prefix'] += 1
00229             line = self.f.next()
00230         else:
00231             if self.is_descline(line):
00232                 self.stat['descline'] += 1
00233                 if self.synth:
00234                     line=self.synth + self.delimiter + self.descline(line)
00235                 self.cols = line.rstrip().split(self.delimiter)
00236             else:
00237                 self.stat['payload'] += 1
00238                 if self.synth: 
00239                     line = "%d" % (len(self) - 1) + self.delimiter + self.clean(line) 
            return line
def DybDbi::csvrw::Source::__iter__ (   self)

Definition at line 240 of file csvrw.py.

00241                       :
00242         return self

def DybDbi::csvrw::Source::__repr__ (   self)

Definition at line 243 of file csvrw.py.

00244                       :
00245         return "Source stat:%r \n cols:%r  " % ( self.stat, self.cols ) 
00246 


Member Data Documentation

Definition at line 181 of file csvrw.py.

Definition at line 181 of file csvrw.py.

Definition at line 181 of file csvrw.py.

Definition at line 181 of file csvrw.py.

Definition at line 181 of file csvrw.py.

Definition at line 181 of file csvrw.py.

Definition at line 181 of file csvrw.py.

skip lines beginning with the prefix

Definition at line 181 of file csvrw.py.


The documentation for this class was generated from the following file:
| Classes | Job Modules | Data Objects | Services | Algorithms | Tools | Packages | Directories | Tracs |

Generated on Fri May 16 2014 09:57:24 for DybDbi by doxygen 1.7.4