/search.css" rel="stylesheet" type="text/css"/> /search.js">
| Classes | Job Modules | Data Objects | Services | Algorithms | Tools | Packages | Directories | Tracs |

In This Package:

Classes | Public Member Functions | Private Types | Private Member Functions | Private Attributes
DybOutputFileRoller Class Reference

Algorithm to control splitting of output files. More...

#include <DybOutputFileRoller.h>

Collaboration diagram for DybOutputFileRoller:
Collaboration graph
[legend]

List of all members.

Classes

struct  ConditionData

Public Member Functions

 DybOutputFileRoller (const std::string &name, ISvcLocator *svcloc)
virtual ~DybOutputFileRoller ()
StatusCode initialize ()
StatusCode execute ()
StatusCode finalize ()

Private Types

typedef std::map< std::string,
std::string > 
ConditionPropertyMap_t
 Note: this is not a property.
typedef std::map< std::string,
ConditionData
CDMap_t

Private Member Functions

StatusCode roll (std::string stream, ConditionData &cd)
StatusCode check_e (std::string stream, ConditionData &cd)
StatusCode check_m (std::string stream, ConditionData &cd)
StatusCode check_s (std::string stream, ConditionData &cd)

Private Attributes

std::string m_condition
 Property Condition: set the default size of output files for any streams not explicitly listed in the ConditionMap.
ConditionPropertyMap_t m_condpropmap
int m_current_execnum
IRootIOSvcm_rio
IRootIOSvc::FileMap m_filemap
CDMap_t m_cdmap

Detailed Description

Algorithm to control splitting of output files.

This works by modifying the RootIOCnvSvc's output file map when a condition is met. If no output files are specified this algorithm quietly does nothing.

The condition is set with the Condition properties, eg 15S will produce files with fifteen seconds worth of data.

The output file map specified on the nuwa.py command line can contain a format code like "%d" (or variants like "%04d"). If so, it will be expanded to hold a count, starting from 1, of the file. If no "%d" code is found then the file names will be modified to change ".root" to "_%04d.root".

This algorithm will roll output file names based on a given condition which consists of integer number followed by a character code:

E: execution cycles M: mebibytes (10^6 bytes) S: seconds period of time as measured by the RegistrationSequences

Note, current RootIO requires all output files to be split on common boundaries in order to be read in correctly.

Brett Viren <bv@bnl.gov> Fri Jan 14 15:30:36 2011

Definition at line 45 of file DybOutputFileRoller.h.


Member Typedef Documentation

typedef std::map<std::string,std::string> DybOutputFileRoller::ConditionPropertyMap_t [private]

Note: this is not a property.

I can be turned on when/if RootIO can handle input streams that do not have common file boundaries. See #671.

ConditionMap: explicitly set condition to split specific output stream. A special stream named "default" can be set which will have the behavior of the Condition property (which overides if set). The map key is a stream name (like "/Event/Gen/GenEvent") and the map value is a condition as described in the Condition property's documentation. Note: for a non-default condition to be applied the stream->file mapping must be explicitly defined in RootIOCnvSvc (via nuwa.py's -O/--output-streams option).

Definition at line 82 of file DybOutputFileRoller.h.

typedef std::map<std::string,ConditionData> DybOutputFileRoller::CDMap_t [private]

Definition at line 105 of file DybOutputFileRoller.h.


Constructor & Destructor Documentation

DybOutputFileRoller::DybOutputFileRoller ( const std::string &  name,
ISvcLocator *  svcloc 
)

Definition at line 14 of file DybOutputFileRoller.cc.

    : GaudiAlgorithm(name,svcloc)
    , m_current_execnum(0)
    , m_rio(0)
{
    declareProperty("Condition",m_condition="",
                    "Set the default condition for rolling to a new output file");

    // This condition map will be a trivial one just containing
    // "default" as set by the global condition above.  It can be made
    // non-trivial when/if RootIO can handle reading input streams
    // with non-common boundaries.  See #671.

    //declareProperty("ConditionMap",m_condpropmap,
    //                "Set the condition map for rolling each stream to a new output file");
}
DybOutputFileRoller::~DybOutputFileRoller ( ) [virtual]

Definition at line 32 of file DybOutputFileRoller.cc.

{
}    

Member Function Documentation

StatusCode DybOutputFileRoller::initialize ( )

Definition at line 36 of file DybOutputFileRoller.cc.

{
    StatusCode sc = this->GaudiAlgorithm::initialize();
    if (sc.isFailure()) return sc;

    // Get IRootIOSvc
    sc = this->service("RootIOCnvSvc",m_rio);
    if (sc.isFailure()) return sc;

    m_filemap = m_rio->fileMap();
    if (!m_filemap.size()) {
        return StatusCode::SUCCESS; // we just won't play this time
    }
    
    // munge the output filenames where needed
    {
        IRootIOSvc::FileMap::iterator it, done = m_filemap.end();
        for (it = m_filemap.begin(); it != done; ++it) {
            string fname = Form(it->second.c_str(),0);
            if (fname != it->second) {
                continue;           // already has a %d code that does something
            }
            
            string::size_type ind = it->second.rfind(".");
            // shouldn't happen, but whatevs, we'll roll with it
            if (string::npos == ind) { 
                ind = it->second.size(); 
            }
            
            // replace filename with one that has a %d code.
            fname = it->second.substr(0,ind) + "_%04d" + it->second.substr(ind);
            it->second = fname;
        }
    }
    
    // specifically set default condition if one is given
    if (m_condition.size()) {
        m_condpropmap["default"] = m_condition;
    }

    { // parse conditions map and strip out any explicit conditions
      // with no matching entry in output file map
        ConditionPropertyMap_t::iterator it, done = m_condpropmap.end();
        for (it=m_condpropmap.begin(); it != done; ++it) {
            string stream = it->first;
            string condition = it->second;

            if (stream != "default") {
                IRootIOSvc::FileMap::iterator fit = m_filemap.find(stream);
                if (fit == m_filemap.end()) {
                    error() << "Condition \"" << condition 
                            << "\" for stream \"" << stream 
                            << "\" has no output file.  " 
                            << "An entry can be specified with nuwa.py's"
                            << " -O/--output-stream option"
                            << endreq;
                    return StatusCode::FAILURE;
                }
            }
            
            string::size_type clen = condition.size();
            if (!clen) {
                error() << "Given a null condition for stream \""
                        << stream << "\"" << endreq;
                return StatusCode::FAILURE;
            }
            int size = atoi(condition.substr(0,clen-1).c_str());
            char unit = condition[clen-1];
            ConditionData cd = ConditionData(size,unit);
            cd.cycles = size;   // special init, count down on this if an E condition
            m_cdmap[stream] = cd;
            info() << "Condition for stream " << stream << ": " << size << unit << endreq;
            sc = this->roll(stream,m_cdmap[stream]);
            if (sc.isFailure()) return sc;
        }
    }

    return StatusCode::SUCCESS;
}
StatusCode DybOutputFileRoller::execute ( )

Definition at line 205 of file DybOutputFileRoller.cc.

{
    if (!m_filemap.size()) {    // no output files, bail blissfully
        return StatusCode::SUCCESS;
    }

    RegistrationSequence* rs = get<RegistrationSequence>(RegistrationSequence::defaultLocation());
    TimeStamp now = rs->earliest();
    ++m_current_execnum;
    info() << "Execution #" << m_current_execnum << " @ " << now << endreq;

    CDMap_t::iterator it, done = m_cdmap.end();
    for (it=m_cdmap.begin(); it != done; ++it) {
        string stream = it->first;
        ConditionData& cd = it->second;

        StatusCode sc;

        switch (cd.unit) {
        case 'M':
            sc = check_m(stream,cd);
            break;
        case 'E':
            sc = check_e(stream,cd);
            break;
        case 'S':
            sc = check_s(stream,cd);
            break;
        default:
            error() << "Unknown condition unit '" << cd.unit << "'" 
                    << " for stream \"" << stream << "\""
                    << endreq;
            sc = StatusCode::FAILURE;
        }
        if (sc.isFailure()) return sc;
    }
    return StatusCode::SUCCESS;
}
StatusCode DybOutputFileRoller::finalize ( )

Definition at line 244 of file DybOutputFileRoller.cc.

{
    return this->GaudiAlgorithm::finalize();
}
StatusCode DybOutputFileRoller::roll ( std::string  stream,
ConditionData cd 
) [private]

Definition at line 115 of file DybOutputFileRoller.cc.

{
    ++cd.count;

    IRootIOSvc::FileMap::iterator fit = m_filemap.find(stream);
    if (fit == m_filemap.end()) {
        error() << "Failed to find output filespec for \"" << stream << "\"" << endreq;
        return StatusCode::FAILURE;
    }

    string fname = Form(fit->second.c_str(),cd.count);
    m_rio->associateOutput(fname.c_str(),stream.c_str());

    info() << "Rolled " << stream << " to file " << fname << endreq;

    return StatusCode::SUCCESS;
}
StatusCode DybOutputFileRoller::check_e ( std::string  stream,
ConditionData cd 
) [private]

Definition at line 161 of file DybOutputFileRoller.cc.

{
    info() << "check_e " << stream << " " << cd.cycles << " " << (void*)&cd << endreq;

    // check for first time
    if (cd.cycles < 0) {
        cd.cycles = cd.size;
    }

    if (cd.cycles>0) {
        --cd.cycles;
        return StatusCode::SUCCESS;
    }
    cd.cycles = cd.size-1;        // reset countdown
    info () << "Reached " << cd.size << " execution cycles" << endreq;
    return this->roll(stream,cd);
}
StatusCode DybOutputFileRoller::check_m ( std::string  stream,
ConditionData cd 
) [private]

Definition at line 133 of file DybOutputFileRoller.cc.

{
    IRootIOSvc::FileMap fm = m_rio->fileMap();
    IRootIOSvc::FileMap::iterator fit = fm.find(stream);
    if (fit == fm.end()) {
        error() << "Failed to find an output file for stream \"" << stream << "\""
                << endreq;
        return StatusCode::FAILURE;
    }
    string fname = fit->second;
    struct stat si;
    int err = stat(fname.c_str(), &si);
    if (err) {
        warning() << "Failed to stat " << fname
                  << ".  Maybe no output yet?  Letting it go."
                  << endreq;
        return StatusCode::SUCCESS;
    }
        
    if (si.st_size/1000000 < cd.size) {  // bytes to mibibytes
        return StatusCode::SUCCESS;
    }
    info () << "Reached " << cd.size << " MiB size" << endreq;
    return this->roll(stream,cd);
}
StatusCode DybOutputFileRoller::check_s ( std::string  stream,
ConditionData cd 
) [private]

Definition at line 181 of file DybOutputFileRoller.cc.

{
    RegistrationSequence* rs = 
        get<RegistrationSequence>(RegistrationSequence::defaultLocation());

    TimeStamp now = rs->earliest();

    // check for first time
    if (cd.timestamp == TimeStamp::GetBOT()) {
        cd.timestamp = now;
        return StatusCode::SUCCESS;
    }

    TimeStamp dt = now - cd.timestamp;
    int deltat = (int)((double)dt);
    if (deltat < cd.size) {     // not yet
        return StatusCode::SUCCESS;
    }
    
    cd.timestamp = now;
    info () << "Reached " << cd.size << " seconds at " << now << endreq;
    return this->roll(stream,cd);
}

Member Data Documentation

std::string DybOutputFileRoller::m_condition [private]

Property Condition: set the default size of output files for any streams not explicitly listed in the ConditionMap.

This should be an integer followed by a unit code. The unit code can be one of:

E: execution cycles M: mebibytes (10^6 bytes) S: seconds period of time as measured by the RegistrationSequences

A special condition "None" will leave the stream untouched.

Definition at line 67 of file DybOutputFileRoller.h.

Definition at line 83 of file DybOutputFileRoller.h.

Definition at line 86 of file DybOutputFileRoller.h.

Definition at line 87 of file DybOutputFileRoller.h.

Definition at line 88 of file DybOutputFileRoller.h.

Definition at line 106 of file DybOutputFileRoller.h.


The documentation for this class was generated from the following files:
| Classes | Job Modules | Data Objects | Services | Algorithms | Tools | Packages | Directories | Tracs |

Generated on Fri May 16 2014 10:15:57 for DybIO by doxygen 1.7.4