/search.css" rel="stylesheet" type="text/css"/> /search.js">
| Classes | Job Modules | Data Objects | Services | Algorithms | Tools | Packages | Directories | Tracs |

In This Package:

Classes | Public Member Functions | Static Public Member Functions | Private Types | Private Member Functions | Private Attributes
OutputFileRollerTool Class Reference

Change output file name when a condition is met. More...

#include <OutputFileRollerTool.h>

Inheritance diagram for OutputFileRollerTool:
Inheritance graph
[legend]
Collaboration diagram for OutputFileRollerTool:
Collaboration graph
[legend]

List of all members.

Classes

struct  ConditionData

Public Member Functions

 OutputFileRollerTool (const std::string &type, const std::string &name, const IInterface *parent)
virtual ~OutputFileRollerTool ()
virtual StatusCode visit (DayaBay::RegistrationSequence &rs)

Static Public Member Functions

static const InterfaceID & interfaceID ()

Private Types

typedef std::map< std::string,
std::string > 
ConditionPropertyMap_t
 Note: this is not a property.
typedef std::map< std::string,
ConditionData
CDMap_t

Private Member Functions

virtual StatusCode _initialize ()
StatusCode roll (std::string stream, ConditionData &cd)
StatusCode check_e (std::string stream, ConditionData &cd)
StatusCode check_m (std::string stream, ConditionData &cd)
StatusCode check_s (std::string stream, ConditionData &cd)

Private Attributes

std::string m_condition
 Property Condition: set the default size of output files for any streams not explicitly listed in the ConditionMap.
int m_startOffset
 Property StartOffset: sets the starting offset for "subrun" numbers to apply to the rolled output files.
ConditionPropertyMap_t m_condpropmap
int m_current_execnum
IRootIOSvc * m_rio
IRootIOSvc::FileMap m_filemap
CDMap_t m_cdmap
bool m_initialized

Detailed Description

Change output file name when a condition is met.

This works by modifying the RootIOCnvSvc's output file map when a condition is met. If no output files are specified this algorithm quietly does nothing.

The condition is set with the Condition properties, eg 15S will produce files with fifteen seconds worth of data.

The output file map specified on the nuwa.py command line can contain a format code like "%d" (or variants like "%04d"). If so, it will be expanded to hold a count, starting from 1, of the file. If no "%d" code is found then the file names will be modified to change ".root" to "_%04d.root".

This algorithm will roll output file names based on a given condition which consists of integer number followed by a character code:

E: execution cycles M: mebibytes (10^6 bytes) S: seconds period of time as measured by the RegistrationSequences

Note, current RootIO requires all output files to be split on common boundaries in order to be read in correctly.

Brett Viren <bv@bnl.gov> Mon Feb 14 17:33:01 2011

Definition at line 47 of file OutputFileRollerTool.h.


Member Typedef Documentation

typedef std::map<std::string,std::string> OutputFileRollerTool::ConditionPropertyMap_t [private]

Note: this is not a property.

I can be turned on when/if RootIO can handle input streams that do not have common file boundaries. See #671.

ConditionMap: explicitly set condition to split specific output stream. A special stream named "default" can be set which will have the behavior of the Condition property (which overides if set). The map key is a stream name (like "/Event/Gen/GenEvent") and the map value is a condition as described in the Condition property's documentation. Note: for a non-default condition to be applied the stream->file mapping must be explicitly defined in RootIOCnvSvc (via nuwa.py's -O/--output-streams option).

Definition at line 91 of file OutputFileRollerTool.h.

typedef std::map<std::string,ConditionData> OutputFileRollerTool::CDMap_t [private]

Definition at line 114 of file OutputFileRollerTool.h.


Constructor & Destructor Documentation

OutputFileRollerTool::OutputFileRollerTool ( const std::string &  type,
const std::string &  name,
const IInterface *  parent 
)

Definition at line 15 of file OutputFileRollerTool.cc.

    : GaudiTool(type,name,parent)
    , m_current_execnum(0)
    , m_rio(0), m_initialized(false)
{
    declareInterface<IRegSeqVisitor>(this);

    declareProperty("Condition",m_condition="",
                    "Set the default condition for rolling to a new output file");

    declareProperty("StartNumber", m_startOffset=0,
                    "Set the starting file number offset, def=0 (first file gets 1).");

    // This condition map will be a trivial one just containing
    // "default" as set by the global condition above.  It can be made
    // non-trivial when/if RootIO can handle reading input streams
    // with non-common boundaries.  See #671.

    //declareProperty("ConditionMap",m_condpropmap,
    //                "Set the condition map for rolling each stream to a new output file");
}
OutputFileRollerTool::~OutputFileRollerTool ( ) [virtual]

Definition at line 39 of file OutputFileRollerTool.cc.

{
}

Member Function Documentation

StatusCode OutputFileRollerTool::visit ( DayaBay::RegistrationSequence rs) [virtual]

Implements IRegSeqVisitor.

Definition at line 212 of file OutputFileRollerTool.cc.

{
    if (!m_initialized) {
        // have to do explicit initialize as ToolSvc and thus this
        // tool gets initialized before RootIOCnvSvc does.
        this->_initialize();
    }

    if (!m_filemap.size()) {    // no output files, bail blissfully
        return StatusCode::SUCCESS;
    }

    TimeStamp now = rs.earliest();
    ++m_current_execnum;
    debug() << "Execution #" << m_current_execnum << " @ " << now << endreq;

    CDMap_t::iterator it, done = m_cdmap.end();
    for (it=m_cdmap.begin(); it != done; ++it) {
        string stream = it->first;
        ConditionData& cd = it->second;

        StatusCode sc;

        switch (cd.unit) {
        case 'M':
            sc = check_m(stream,cd);
            break;
        case 'E':
            sc = check_e(stream,cd);
            break;
        case 'S':
            sc = check_s(stream,cd);
            break;
        default:
            error() << "Unknown condition unit '" << cd.unit << "'" 
                    << " for stream \"" << stream << "\""
                    << endreq;
            sc = StatusCode::FAILURE;
        }
        if (sc.isFailure()) return sc;
    }
    return StatusCode::SUCCESS;
}
StatusCode OutputFileRollerTool::_initialize ( ) [private, virtual]

Definition at line 43 of file OutputFileRollerTool.cc.

{
    m_initialized = true;

    // Get IRootIOSvc
    StatusCode sc = this->service("RootIOCnvSvc",m_rio);
    if (sc.isFailure()) return sc;

    m_filemap = m_rio->fileMap();
    if (!m_filemap.size()) {
        info () << "Output filemap is empty, nothing for me to do." << endreq;
        return StatusCode::SUCCESS; // we just won't play this time
    }
    
    // munge the output filenames where needed
    {
        IRootIOSvc::FileMap::iterator it, done = m_filemap.end();
        for (it = m_filemap.begin(); it != done; ++it) {
            string fname = Form(it->second.c_str(),0);
            if (fname != it->second) {
                continue;           // already has a %d code that does something
            }
            
            string::size_type ind = it->second.rfind(".");
            // shouldn't happen, but whatevs, we'll roll with it
            if (string::npos == ind) { 
                ind = it->second.size(); 
            }
            
            // replace filename with one that has a %d code.
            fname = it->second.substr(0,ind) + "_%04d" + it->second.substr(ind);
            it->second = fname;
        }
    }
    
    // specifically set default condition if one is given
    if (m_condition.size()) {
        m_condpropmap["default"] = m_condition;
    }

    { // parse conditions map and strip out any explicit conditions
      // with no matching entry in output file map
        ConditionPropertyMap_t::iterator it, done = m_condpropmap.end();
        for (it=m_condpropmap.begin(); it != done; ++it) {
            string stream = it->first;
            string condition = it->second;

            if (stream != "default") {
                IRootIOSvc::FileMap::iterator fit = m_filemap.find(stream);
                if (fit == m_filemap.end()) {
                    error() << "Condition \"" << condition 
                            << "\" for stream \"" << stream 
                            << "\" has no output file.  " 
                            << "An entry can be specified with nuwa.py's"
                            << " -O/--output-stream option"
                            << endreq;
                    return StatusCode::FAILURE;
                }
            }
            
            string::size_type clen = condition.size();
            if (!clen) {
                error() << "Given a null condition for stream \""
                        << stream << "\"" << endreq;
                return StatusCode::FAILURE;
            }
            int size = atoi(condition.substr(0,clen-1).c_str());
            char unit = condition[clen-1];
            ConditionData cd = ConditionData(size,unit);
            cd.cycles = size;   // special init, count down on this if an E condition
            m_cdmap[stream] = cd;
            info() << "Condition for stream " << stream << ": " << size << unit << endreq;
            sc = this->roll(stream,m_cdmap[stream]);
            if (sc.isFailure()) return sc;
        }
    }

    return StatusCode::SUCCESS;
}
StatusCode OutputFileRollerTool::roll ( std::string  stream,
ConditionData cd 
) [private]

Definition at line 122 of file OutputFileRollerTool.cc.

{
    ++cd.count;

    IRootIOSvc::FileMap::iterator fit = m_filemap.find(stream);
    if (fit == m_filemap.end()) {
        error() << "Failed to find output filespec for \"" << stream << "\"" << endreq;
        return StatusCode::FAILURE;
    }

    string fname = Form(fit->second.c_str(),cd.count+m_startOffset);
    m_rio->associateOutput(fname.c_str(),stream.c_str());

    info() << "Rolled " << stream << " to file " << fname << endreq;

    return StatusCode::SUCCESS;
}
StatusCode OutputFileRollerTool::check_e ( std::string  stream,
ConditionData cd 
) [private]

Definition at line 168 of file OutputFileRollerTool.cc.

{
    debug() << "check_e " << stream << " " << cd.cycles << " " << (void*)&cd << endreq;

    // check for first time
    if (cd.cycles < 0) {
        cd.cycles = cd.size;
    }

    if (cd.cycles>0) {
        --cd.cycles;
        return StatusCode::SUCCESS;
    }
    cd.cycles = cd.size-1;        // reset countdown
    info () << "Reached " << cd.size << " execution cycles" << endreq;
    return this->roll(stream,cd);
}
StatusCode OutputFileRollerTool::check_m ( std::string  stream,
ConditionData cd 
) [private]

Definition at line 140 of file OutputFileRollerTool.cc.

{
    IRootIOSvc::FileMap fm = m_rio->fileMap();
    IRootIOSvc::FileMap::iterator fit = fm.find(stream);
    if (fit == fm.end()) {
        error() << "Failed to find an output file for stream \"" << stream << "\""
                << endreq;
        return StatusCode::FAILURE;
    }
    string fname = fit->second;
    struct stat si;
    int err = stat(fname.c_str(), &si);
    if (err) {
        warning() << "Failed to stat " << fname
                  << ".  Maybe no output yet?  Letting it go."
                  << endreq;
        return StatusCode::SUCCESS;
    }
        
    if (si.st_size/1000000 < cd.size) {  // bytes to mibibytes
        return StatusCode::SUCCESS;
    }
    info () << "Reached " << cd.size << " MiB size" << endreq;
    return this->roll(stream,cd);
}
StatusCode OutputFileRollerTool::check_s ( std::string  stream,
ConditionData cd 
) [private]

Definition at line 188 of file OutputFileRollerTool.cc.

{
    RegistrationSequence* rs = 
        get<RegistrationSequence>(RegistrationSequence::defaultLocation());

    TimeStamp now = rs->earliest();

    // check for first time
    if (cd.timestamp == TimeStamp::GetBOT()) {
        cd.timestamp = now;
        return StatusCode::SUCCESS;
    }

    TimeStamp dt = now - cd.timestamp;
    int deltat = (int)((double)dt);
    if (deltat < cd.size) {     // not yet
        return StatusCode::SUCCESS;
    }
    
    cd.timestamp = now;
    info () << "Reached " << cd.size << " seconds at " << now << endreq;
    return this->roll(stream,cd);
}

Member Data Documentation

std::string OutputFileRollerTool::m_condition [private]

Property Condition: set the default size of output files for any streams not explicitly listed in the ConditionMap.

This should be an integer followed by a unit code. The unit code can be one of:

E: execution cycles M: mebibytes (10^6 bytes) S: seconds period of time as measured by the RegistrationSequences

A special condition "None" will leave the stream untouched.

Definition at line 71 of file OutputFileRollerTool.h.

Property StartOffset: sets the starting offset for "subrun" numbers to apply to the rolled output files.

Default is 0, making the numbering start at 1.

Definition at line 76 of file OutputFileRollerTool.h.

Definition at line 92 of file OutputFileRollerTool.h.

Definition at line 95 of file OutputFileRollerTool.h.

IRootIOSvc* OutputFileRollerTool::m_rio [private]

Definition at line 96 of file OutputFileRollerTool.h.

IRootIOSvc::FileMap OutputFileRollerTool::m_filemap [private]

Definition at line 97 of file OutputFileRollerTool.h.

Definition at line 115 of file OutputFileRollerTool.h.

Definition at line 117 of file OutputFileRollerTool.h.


The documentation for this class was generated from the following files:
| Classes | Job Modules | Data Objects | Services | Algorithms | Tools | Packages | Directories | Tracs |

Generated on Fri May 16 2014 10:08:03 for DybTool by doxygen 1.7.4