Navigation
index
modules
|
next
|
Offline User Manual 22909 documentation
»
Quick search
Enter search terms or a module, class or function name.
Other index
BNL
NUU
NTU
Table Of Contents
Daya Bay Offline User Manual
Unrecognized latex commands
Indices and tables
Next topic
Introduction
This Page
Show Source
Daya Bay Links
dybsvn
dybaux
BNL wiki
OUM at
BNL
,
NUU
,
HKU
Doxygen at
BNL
,
NUU
,
HKU
ROOT Docs at
CERN
,
NUU
Content Skeleton
Introduction
Quick Start
Analysis Basics
Offline Infrastructure
Installation and Working with the Source Code
Offline Framework
Data Model
Data I/O
Detector Description
Kinematic Generators
Detector Simulation
Electronics Simulation
Trigger Simulation
Readout
Simulation Processing Models
Reconstruction
Database
Database Maintanence
Bibliography
Testing Code With Nose
Standard Operating Procedures
Administrative Operating Procedures for SVN/Trac/MySQL
Python API
Documentation
Daya Bay Offline User Manual
¶
Version:
22909
Date:
May 16, 2014
PDF:
OfflineUserManual.pdf
(via reStructuredText and Sphinx)
Old PDF:
main.pdf
(direct from latex)
Introduction
Intended Audience
Document Organization
Contributing
Building Documentation
Typographical Conventions
Quick Start
Offline Infrastructure
Installation and Working with the Source Code
Installing a Release
Using an existing release
Projects
Packages
Environment
Offline Framework
Data Model
Detector Description
Kinematic Generators
Detector Simulation
Quick Start with Truth Information
Particle History
Unobservable Statistics
Configuring Truth Information
Configuring Particle Histories
Configuring Unobservable Statistics
Accessing the resulting truth information
Electronics Simulation
Trigger Simulation
Readout
Event Display
A Plain Event Display: EvtDsp
Simple Mode
Advance Mode
Reconstruction
Database
Analysis Basics
Introduction
Daya Bay Data Files
Opening data files
Histogramming data
Histogramming Raw DAQ data
Some ROOT Tree Tricks
Analysis Examples (or A Treatise on Cat-skinning)
dybTreeDraw.C
dybGetLeaf.C
dybTreeSetBranch.C
dybNuWaHist.py
Advanced Examples
Combining ‘Unfriendly’ Trees
Using
TTree::Draw()
with ‘Unfriendly’ Trees
NuWa Basics
The
nuwa.py
Command
NuWa Job Modules
NuWa Recipes
See the history of a NuWa File
Tag Events in a NuWa File
Add Variables to a NuWa File
Adding User-defined Variables to Tagged Events
Copy Data Paths to a New File
Write Tagged Data to a New File
Change an Existing Job Module
Write a Python analysis Algorithm
Write a C++ analysis Algorithm
Modify Part of NuWa
Using Services
Cheat Sheets
Loading the NuWa software
Installing the NuWa software
Making your own Project
Standard Data Files
Using the Catalog
Data File Contents
Common NuWa Commands
Conventions and Context
Sites
Detectors
Raw DAQ Data
Conversion from
.data
Raw data in ROOT
Readout data in NuWa
Calibrated Data
Calibrated data in ROOT
Calibrated data in NuWa
Calibrated Statistics Data
Calibrated statistics data in ROOT
Calibrated statistics data in NuWa
Reconstructed Data
Reconstructed data in ROOT
Reconstructed data in NuWa
Spallation Data
Spallation data in ROOT
Spallation data in NuWa
Coincidence Data
Coincidence data in ROOT
Coincidence data in NuWa
NuWa Services
Computer Clusters
Miscellaneous
Time Axes in ROOT
Hands-on Exercises
Offline Infrastructure
Mailing lists
DocDB
Wikis
Trac bug tracker
Installation and Working with the Source Code
Using pre-installed release
Basic setup
Setup the
dybgaudi
project
Instalation of a Release
Anatomy of a Release
Release, Projects and Packages
Personal Projects
Version Control Your Code
Using SVN to Contribute to a Release
Using GIT with SVN
Technical Details of the Installation
LCGCMT
Builders
Offline Framework
Introduction
Framework Components and Interfaces
Common types of Components
Algorithms
Tools
Services
Writing your own component
Algorithms
Tools
Services
Generalized Components
Properties and Configuration
Overview of configuration mechanism
Configurables
Configurables and Their Names
Naming Gaudi Tool Configurables
The Package
Configure
Class and Optional Helper Classes
User Job Option Scripts
User Job Option Modules
The
nuwa.py
main script
Example: Configuring
DetSimValidation
Data Model
Overview
HeaderObject
Times
Absolute Time
Relative Time
Reference times
Sub-object precision times
Time Extents
How Some Times are Defined
Examples of using the Data Model objects
Tutorial examples
Data I/O
Goal
Features
Packages
I/O Related Job Configuration
How the I/O Subsystem Works
Execution Cycle vs. Event
Registration Sequence
Writing data out
When to write out
Reading data in
Adding New Data Classes
Class Locations and Naming Conventions
Guidelines for Writing Persistent Data Classes
Steps to Follow
Difficulties with Persistent Data Classes
Writing Converters
Converter Header File
Converter Implementation File
Register Converter with Gaudi
Resolving references
CMT
requirements
File
Detector Description
Introduction
Volumes
Logical
Physical
Touchable
Conventions
AD PMTs
Pool PMTs
RPC
Coordinate System
XML Files
Transient Detector Store
Configuring the Detector Description
PMT Lookups
Visualization
Kinematic Generators
Introduction
Generator output
Generator Tools
Generator Packages
Types of GenTools
GenTools package
GenDecay Package
Configuration
Configurables
GenDecay Configurables
GenTools.Configure
Gnrtr.Configure
and its Stages
GenTools Dumper Algorithm
GenTools Job Option Modules
MuonProphet
Motivation
Generation Mechanism
Code Organisation
Configuration
Output
Trigger Bits
Quick Start
Detector Simulation
Introduction
Configuring DetSim
Truth Information
Particle Histories
Particle History Data Objects
Creation Rules
Examples, Tips, Tricks
Unobservable Statistics
Description
“Unobservable” Statistic Objects
Creation Rules
Examples, Tips, Trucks
Parameter Reference
The DrawHistoryAlg Algorithm
Truth Parameters
Electronics Simulation
Introduction
Algorithms
Tools
Hit Tools
FEE Tool: EsIdealFeeTool
Multiplicity Generation and TDC
ADC Generation
Simulation Constant
Trigger Simulation
Introduction
Configuration
Current Triggers
TsMultTriggerTool
TsExternalTriggerTool
Adding a new Trigger
Readout
Introduction
ReadoutHeader
SimReadoutHeader
Readout Algorithms
ROsSequencerAlg
ROsReadoutAlg
Readout Tools
ROsFeeReadoutTool
ROsFecReadoutTool
ROsFeeAdcMultiTool
ROsFeeAdcPeakOnlyTool
ROsFeeTdcTool
Simulation Processing Models
Introduction
Fifteen
Quick Start
Simulation Stage
Stage Tool
Gnrtr
DetSimProc
ElecSimProc
TrigReadProc
SingleLoader
LoadingProc
Algorithm Sim15
Customize Your Simulation Job
A General Example
Example for LoadingProc
Reminders and Some Common Errors
Reconstruction
Database
Database Interface
Concepts
Types of Data
Simple, Compound and Aggregated
Tables of Data
A Cascade of Databases
Context Sensitive
Extended Context
SimFlag Association
Authorising Databases and Global Sequence Numbers
Validity Management
Rollback
Lightweight Pointers to Heavyweight Data
Natural Table Index
Task
Sub-Site
Level 2 (disk) Cache
Running
Setting up the Environment
Configuring
Rollback
MakeConnectionsPermanent
Ordering Context Query Results
Level 2 Cache
Output Level
Accessing Existing Tables
Introduction
Accessing Detector Descriptions
Making the Query
Accessing the Results Table
Getting Data from a Row
Making Further Queries
Simple Optimisation
Extended Context Queries
Making the Query
Accessing the Results Table
Error Handling
Response to Errors
Error Logging
Creating New Tables
Choosing Table Names
Creating Detector Descriptions
A Simple Example
Creating a Database Table
Designing a Table Row Class
The Dictionary files
Data Encapsulation
Filling Tables
Overview
Aggregate Numbers
Simple Example
Using DbiWriter
Table Row Responsibilities
Creating and Writing Temporary Tables
Clearing the Cache
ASCII Flat Files and Catalogues
Overview
Flat Files
Format
URL
Catalogues
Format
URL
Example
MySQL Crib
Introduction
Installing
Running mysql
System Administration
Selecting a Database
Creating Tables
Filling Tables
Making Queries
Performance
Holding Open Connections
Truncated Validity Ranges
Timing
Database Maintanence
Introduction
Building and Running dbmjob
Building
Running
Exporting Data
Importing Data
Bibliography
Testing Code With Nose
Nosetests Introduction
Unit Testing Philosophy
Examples
DybDbi
DBUpdate
DbiTest
Recommendations
short and focussed
dont repeat yourself
Zero Cost Test Development
References
External
doc:6280 : Encouraging Nose Testing
doc:5258 : Your NuWa Testing System
doc:3645 : NuWa Offline Software Testing System
doc:3091 : NuWa-Trac and Testing System
Using Test Attributes
Package Level Nosetesting with attributes
Testing at dybinst level
Testing at bitten slave level
Commit Message controlled deep testing
Periodic deep testing based on revision number
Running Tests Using dybinst
Informing dybinst about tests
Getting the slaves to auto run package tests
Testing nose plugins
Setup vitualenv sandbox
Get into the virtualenv
Interesting Plugins
nosepipe
insulate
Standard Operating Procedures
DB Definitions
Database Topology Diagram
offline_db
replication data flow
Database names
DBI Very Briefly
Validity Tables
Validity Table Timestamps
How these times fit in
Choosing Validity Ranges
Rollback and Production
Using Rollback to Debug/Workaround problem DB entries
What is
TASK
for ?
Features of Clean Validity Tables
Overlay Versioning
DBI Q and A
Doesnt
TIMEEND
of
EOT
overshadow valid entries when we correct an earlier entry ?
How do we make sure not to end up with
SEQNO
gaps ?
If my update has a given SEQNO in my
tmp_offline_db
, will it have the same in the
offline_db
?
What are fastforward commits ? Why are they needed ?
Or is the
offline_db
smart so that it automatically gives it the next number in the sequence ?
How can
SEQNO
be missed?
Rules for Code that writes to the Database
Scope of Rules
DB Writing Code Management
Rationale for dybsvn rule
Recording how DB updates were prepared
Verification of reproducibility
Testing DB writing code
Configuring DB Access
Create Simple DB configuration file
Standardized Section Names
Section dependent testing
DBCONF
envvar
Cascade configuration
Configuring access to ascii catalog
Using dybaux as ascii catalog
N ways to set an envvar
bash
Inferior shells such as tcsh/csh
python
Background Information
What is a mysql dump file ?
What is a DBI ascii catalog ?
Hands-On Exercise 1 : Troubleshooting DB connection configuration
Check with mysql client
Check with
db.py
Check with DBI
DBI error when
DBCONF
not defined
Hands On Exercise 2 : Interactive DybDBI
Get into
ipython
Interactively verify connection
Interactive Exploration with
ipython
TAB completion
DybDbi
with some magic
DB Table Updating Workflow
Objectives
Workflow Outline
Rationale for this workflow
Planning Update Size and Frequency
workflow steps in detail
Copy
offline_db
to
tmp_offline_db
Perform updates and validation on
tmp_offline_db
Early Validations
Communicate updates via SVN repository
Decoupled Updating Workflow
Serialized Updating
How the SVN ascii catalog is primed
Annotating Updates
Pre-commit enforced validation : DBI Gatekeeper
Demonstrate tests and Request Propagation
Database Managers Propagate updates from
dybaux
into
offline_db
Propagation of multiple commits with
dbaux.py
Handling non-propagated dybaux commits
Post-propagation cross check
Quick Validations
Exceptional Operating Procedures for Major Changes
Hands On Exercise 3 : Copy Offline DB
Nosetests of workflow steps
Development History of Workflow
Table Specific Instructions
CalibPmtFineGain
Environment setup
Make temporary copy of
offline_db
Validation and uploading to
tmp_offline_db
Before uploading to
tmp_offline_db
validation
Perform
tmp_offline_db
uploading
Valdation after uploading to
tmp_offline_db
Commit to
dybaux
SVN
Prepare annotation for the update
DB Table Writing
Using
DybDbi
to write to
tmp_offline_db
Configure Target DB
CSV handling
Map CSV fieldnames to DBI attributes
Create
DbiWriter<T>
and set
ContextRange
Convert CSV rows and write to DB
Command line and filename Parsing
Hands On Exercise 4 : Write
$DBWRITERROOT/share/DYB_MC_AD1.txt
into
CalibPmtSpec
Assigning Applicability of Constants
Context Range
Choosing
TIMEEND
Determine run start time from a run number
Overlay Versioning Demonstration
Many more examples of DB writing with
DybDbi
Using
DBWriter
to write to
tmp_offline_db
DB Table Reading
DB Reading with
DybDbi
Default Context Reading
Examine Default Read Context
Change Read Context
Using
mysql
client
Hands On Exercise 5 : Read from DB with varying context
Hands On Exercise 6 : Read run timestart/timeend from
DaqRunInfo
table
Default Context Query
Modify to use wideopen validity context
Fixing this page if it breaks
Debugging unexpected parameters
Configuration check
ipython
DybDbi
session
DbiDataSvc test
mysql client session
Following the mysql tail
GDB Debugging of Template Laden DBI
DB Table Creation
Workflow Outline for Adding Tables
Design Tables
Prepare
.spec
File
Generate Row Classes from
.spec
Ensuring Consistency When Changing Spec
Copy
offline_db
to
tmp_offline_db
Create New Tables in
tmp_offline_db
Populate New Table With Dummy Data
Verify tables using the mysql client
DB Validation
Interactive Checking with ipython
DB Checking
Ascii Catalog Checking
Workflow Checks
Generic Validation
DB Testing
Nosetesting reminder
When and where
Standardized testing
Responsibility for maintaining tests
Responsibility for running tests
Proof of testing
Dealing with mistakes
Checking entries make sense
Status of packages with tests
Comparing python datetimes with DBI TimeStamps
DB Administration
Temporary DB Setup by MySQL Administrators
Custom DB Operations
Tools to manipulate mysqldump files
Preparing and Comparing Dump files
Table renaming in DB
Dump using extended insert
Compare extended insert dumps
communicating dumps via website
Download mysqldump file and load into DB
download dump and verify digest
Checking the dump
Testing loading into
tmp_copy_db
Simple checks on loaded table
Fixup DBI metadata table LOCALSEQNO
Verifying offline_db load by another dump
Copying a few DBI tables between DBs using rdumpcat, rloadcat
Non-decoupled rdumpcat into empty folder
Loading of partial ascii catalog into target DB with fastforwarding of INSERTDATEs
Jumpstarting
offline_db.DqChannelPacked
table
Create mysqldump file
Record size/digest of dump
Check its viable by creating a DB from it
Position that for web accessibility (admin reminder)
Download the dump and check digest
CQScraper testing
Create a test DB to check CQScraper Operation
Load the mysqldump creating the new new tables
Fixup LOCALSEQNO
Configure a node to run the CQScraper cron task
Repeat for
offline_db
DB Services
User Interfaces to DBI Data
Tables which are Missing something
DCS tables grouped/ordered by schema
SAB_TEMP
DBNS_ACU_HV_SlotTemp
DBNS_Temp
AD1_TEMP
DBNS_HALL5_TEMP
config_table
DYBAlarm
DBNS_AD1_HV_Imon
DBNS_AD2_HV_Imon
SAB_AD1_HV_Imon
SAB_AD2_HV_Imon
SAB_AD2_HV_SlotTemp
DBNS_AD1_HV_SlotTemp
DBNS_AD2_HV_SlotTemp
SAB_AD1_HV_SlotTemp
DBNS_SAB_TEMP
site_table
DBNS_MUON_PMT_HV_Imon
DBNS_MUON_PMT_HV_SlotTemp
status_table
DBNS_AD_HV_SlotTemp
dyb_muoncal
DBNS_ACU_HV_Pw
DBNS_ACU_HV_Imon
DBNS_ACU_HV_Vmon
EH1_ENV_RadonMonitor
DBNS_AD1_LidSensor
DBNS_AD2_LidSensor
DBNS_AD1_VME
DBNS_AD2_VME
DBNS_IW_VME
DBNS_Muon_PMT_VME
DBNS_OW_VME
DBNS_RPC_VME
SAB_AD1_VME
DBNS_AD1_HV
DBNS_AD2_HV
SAB_AD1_HV_Vmon
SAB_AD2_HV_Vmon
SAB_AD2_HV_Pw
DBNS_AD1_HVPw
DBNS_AD2_HV_Pw
SAB_AD1_HV_Pw
DBNS_MUON_PMT_HV_Vmon
DBNS_MUON_PMT_HV_Pw
DBNS_AD_HV_Imon
DBNS_AD_HV_Vmon
DBNS_AD_HV_Pw
Non DBI access to DBI and other tables
Summary of Non DBI approaches
Python ORMs (Django, SQLAlchemy)
ROOT
TSQL
High Performance Approaches
SQLAlchemy access to DBI tables with NonDbi
Scraping source databases into
offline_db
Generic Scraping Introduction
Scraper Status
DCS peculiarities
Time zones and scraping
TODO
Framework Level
Specific Regimes
Running Scrapers
Dybinst Level
Package Level
Implementing Scrapers
Outline Steps
Create Scraper Module
Implementing
changed
Implementing
propagate
Generic Aggregation
Error Handling
Configuring Scrapers
Understanding Scraper Operation
Catchup and Sleep Auto-Tuning
Configuration Mechanics
Configuration Tuning
Testing Scraper Operation
Test Scraper With Faker
Faker configuration
Preparing Target DB for testing
Seeding Target Database
Scraper Logging
Continuous running under supervisord
Initial Setup
Supervisorctl CLI
Steps to Deployment
Development Tips
Obtain mysqldump of DCS DB to populate fake source DB
Single table mysqldump for averager testing
Append capable mysqldumps
Multi-source table test
Start from scratch following schema changes to DCS
Interactive SQLAlchemy Querying
DBI Internals
Overlay versioning implementation
Overlay overriding problem
Fix attempt A
Possible Workaround
Possible Solution
Looking for preexisting manifestations
Problem with this fix A
Write single entry into empty table
Write two entries at same validity range ts:EOT into empty table
Write 3 entries for different runs into empty table
Delving into overlay detection and
DbiValidityRecBuilder
fGap : special vrec holding trim results
Trimming in Builder ctor
AndTimeWindow : overlap range
FindTimeBoundaries
Bracketed Trimming : effective range reduced to overlap other
Non bracketed trim : effective range reduced to exclude the other
Double Overlay Example
Trim In full
DBI Overlay Versioning Bug
Background
Root Cause of Issue
Ordering of validity queries
Validity Look Up Tables
DBI Scanning
Current Intended code/SOP changes
Planned rollout of DBI modifications
Extra Ordering Fix
Timestart Flooring and Collision Avoidance
Intended Migration of Existing DB entries
Rebuild Approach
Table Rebuilding and Insertdates
Table Summary
CableMap/HardwareID
CalibPmtSpec
CalibPmtHighGain
Test Rebuilding
Fixup Validity Approach
Current Transfixion Approach
VLUT Comparisons
Summary Examination
tmp_offline_db (copy of offline_db)
fix_offline_db (with VERSIONDATE TIMESTART flooring)
tmp_offline_db_cf_fix_offline_db
Observations
Sampling VLUT extracts
tmp_offline_db_cf_fix_offline_db vlut orderingSEQNOdesc
insertdates:19
timestarts:18
ndif:19
Alternative to Recreat Rather Than Fix Approach
CableMap/HardwareID
Pumping dybaux history with
auxlog.py
Re-creation Discrepancy in FEC loading : resolved with takebogus option
Relevant Tickets
Bogus Logic
Whole table groupby INSERTDATE for overview
DBI Validity Ordering Change
Payload Digest Rather than SEQNO comparison
CableMap
HardwareID
DBI from C++
Primer for C++ usage of DBI
Raw C++ DBI
C++ DybDbi
C++ DybDbi headers
Administrative Operating Procedures for SVN/Trac/MySQL
Tasks Summary
Subversion/Trac/Autobuild/test Support
Overview
Responsibilities Summary
Lin Tao (IHEP)
tasks
Jimmy Ngai (HKU)
tasks
resources
Routine Operations
Subversion/Trac maintenance
Automation Suggestion
Bitten slave autobuild/test system
Automation Suggestion
Maintain dybinst
Offline User Manual Mechanics
Maintain remote monitor node (non-NuWa)
Maintain remote source node
Exceptional Tasks
Debugging bitten build system
Migrations
Bloated Trac SQLite DB
Backup
Offline Database Support Tasks
Overview
Routine Operations
Database Management
Documentation
Maintain SOP tools
Other tools
SOP Policing
DBI/DybDbi Tech Support
Custom Scripts Advice/Help
Custom Operations
Remote NuWa monitor node
Tour of dybgaudi/Database
Exceptional Tasks
Scraper additions
DBI/DybDbi/NonDbi debugging
Database Interventions
Corruption Recovery
Channel Quality DB Maintenance
backup/transfer script
monitoring script
Responsibilties for maintainer
preparatory task
tasks
SVN/Trac
Backups Overview
Availability of Backups
SVN/Trac backups
MySQL backups
Monitoring
crontabs
blyth@cms01.phys.ntu.edu.tw
root@cms01.phys.ntu.edu.tw
root@cms02.phys.ntu.edu.tw
blyth@dayabay.ihep.ac.cn
root@dybdb1.ihep.ac.cn
managed by Qiumei
root@dybdb2.ihep.ac.cn
managed by Qiumei
DbiMonitor package : cron invoked nosetests
DbiMonitor.tests.test_dcs
Checking update age of tables in IHEP DCS mirror
Update Expectations
DB connection config
python module running
dybinst level running
crontab automation
nosetests and logging issue
DbiMonitor.tests.test_offline
Checking update age of tables in offline_db
Technical issue regards logging and nosetests
Env Repository : Admin Infrastructure Sources
Installing Env
Heirarchy of Bash Functions
Node Characterisation
Dybinst : Dayabay Offline Software Installer
Trac+SVN backup/transfer
Backups with
scm-backup-all
Offbox Transfers with
scm-backup-rsync
Recovery using
scm-recover-all
Distributed backup monitoring
Adding a new target node for backups
Node characterization
Placement of SSH keys
Add target tag to
BACKUP_TAG
of source node
SSH Setup For Automated transfers
Debugging Blocked SSH
ssh-agent process monitoring
Offline DB Backup
Backup System
Issues
Monitoring
Crontab Auto Recovery
Interactive Recovery
Table Size Checks
DBSVN : dybaux SVN pre-commit hook
Example of failure
Fabricate pre-commit working copy
Installing into the SVN server
Bitten Debugging
Extract
trac.db
from backup tarball
Examining bitten tables
table counts
bitten_build
MySQL DB Repair
MySQL Table Repair
Check table types and sizes
MySQL Backup/Recovery Tools
mysqldump
mysqlhotcopy
Usage
mysqlhotcopy.py
Corruption incident
tmp_ligs_offline_db
: timeline and next steps
Timeline Summary
Next Steps
Lessons from MySQL corruption incident
Preventive Avoidance
Large DB very different to small ones
Mitigating Pain/Disruption of corruption recovery
TODO: Investigate DBI response to crashed payload OR validity table
SOP for MySQL corruption
Comparing tables sizes of uncompressed and compressed tables
Python API
DB
DybPython.db
TODO
Required Arguments
command summary
former commands
using db.py in standalone manner (ie without NuWa)
checkout offline_db catalog from dybaux
Test usage of serialized ascii DB
rloadcat testing, DB time machine
DybPython.db.DB
DBAUX
DybPython.dbaux
Checks after performing
rloadcat(s)
Logfile Checks
Manual Checks
rloadcat checks in various situations
AVOIDED ISSUES
DybPython.dbaux.Aux
DBConf
DybPython.dbconf
DBConf
DBCas
DybPython.dbcas
dbsvn - DBI SVN Gatekeeper
DybPython.dbsvn
Usage examples
How this script fits into the workflow
What is validated by
dbsvn.py
Rationale behind these validations
Commit denial
OVERRIDE commits
Deployment of pre-commit hook on SVN server
Hook Deployment on server remote from
dybsvn
Typical Problems with the Hook
Trac Config to limit large diff hangs
DBIValidate
DBSRV
DybPython.dbsrv
dbsrv : MySQL Server Utilities
TODO
Usage
Comparing DB via partitioned dump
diffing the dumped partitions
oops a difference, but its just different formatting of 0.0001 or 1e-04
Commands
summary
dumplocal
loadlocal
partitioned loadlocal
Partitioned Commands
partitioned loadlocal
quick partitioning test
Archiving and transfers to remote node
Transfer Optimization
Full archive/transfer cron test from belle7 to belle1
Installation on dybdb2
Prepare target node
Setup mysql config at source
SSH environment configuration
Get DybPython from dybsvn
Quick Interactive Test
CRON commandline
Transfer Monitoring
Obtain the backup tarballs
Partioned dump usage
System python API warning
Import Notes
DybPython.dbsrv.DB
DybDbiPre
Tab
DybDbi
DybDbi
DybDbi.Wrap
DybDbi.CSV
DybDbi.Source
DybDbi.Mapper
DybDbi.Ctx
DybDbi.DbiCtx
When reading from DB
sqlcontext
datasql
Expert Usage Only
Use when writing to the DB
DybDbi.vld.versiondate
DybDbi.vld.vlut
DEFICIENCIES
Production Run From Scratch
Payload Digest Based Comparisons
Presenting the LUTs with Sphinx
Debugging
Nginx Hookup
Machinery Issues
DybDbi.vld.vsmry
DybDbi.IRunLookup
DybDbi.ILookup
DybDbi.AdLogicalPhysical
DybDbi.Context
DybDbi.ContextRange
DybDbi.TimeStamp
DybDbi.ServiceMode
DybDbi.Site
DybDbi.SimFlag
DybDbi.DetectorId
DybDbi.Detector
DybDbi.DetectorSensor
DybDbi.FeeChannelId
DybDbi.FeeHardwareId
DybDbi.PmtHardwareId
DybDbi.Dbi
DybDbi.GPhysAd
DybDbi.GSimPmtSpec
DybDbi.GCalibPmtSpec
DybDbi.GCalibFeeSpec
DybDbi.GFeeCableMap
DybDbi.GDaqRunInfo
DybDbi.GDaqCalibRunInfo
DybDbi.GDaqRawDataFileInfo
DybDbi.GDbiLogEntry
DybDbi.GDcsAdTemp
DybDbi.GDcsPmtHv
DybPython
DybPython.Control
DybPython.dbicnf
DbiDataSvc
DbiDataSvc
NonDbi
NonDbi
SQLAlchemy Ecosystem
Deficiencies
Accessing Non Dbi tables with SQLAlchemy
Accessing DBI pairs with SQLAlchemy
How to add a class/table
References
SQLite tips
Implementation Notes
Scraper
Scraper
Table specific scraper module examples
Scraper.pmthv
Scraper.pmthv.PmtHv
Scraper.pmthv.PmtHvSource
Scraper.pmthv.PmtHvScraper
Scraper.pmthv.PmtHvFaker
Scraper.adtemp
Scraper.adtemp.AdTemp
Scraper.adtemo.AdTempSource
Scraper.adtemp.AdTempScraper
Scraper.adtemp.AdTempFaker
Scrapers in development
Scraper.adlidsensor
Scraper.adlidsensor.AdLidSensor
Scraper.dcs
: source DB naming conventions
Scraper.base
: directly used/subclassed
Scraper.base.main()
Scraper.base.Regime
Scraper.base.DCS
Scraper.base.Scraper
Scraper.base.Target
Scraper.base.Faker
Other classes used internally
Scraper.base.sourcevector.SourceVector
Scraper.base.aparser.AParser
: argparser/configparser amalgam
Scraper.base.parser.Parser
:
Scraper.base.sa.SA
: details of SQLAlchemy connection
DybTest
dybtest
dybtest.histref
dybtest.cfroot
dybtest.capture
Documentation
About This Documentation
Building this documentation
Who needs to build the Sphinx docs ?
Once only virtualenv setup
Installation of Sphinx and converter into virtual python
Additional dependencies
Sphinx Quickstart/Configuration
Building docs
Partial Builds
Possible Problems
Primer on Sphinx/reST Usage
External Links
Intersphinx
Object Inventories
Graphviz Figures
Documentation from docstrings
How to AutoDocifying a module
Creating Useful AutoDocumentation
tips for docstring preparation
docstring debugging
general reST debugging
Doxygen : automated documentation of C++ API
Doxygen Integration with Breathe
Publishing Documentation on Separate Webserver
Configure Target
Test Dybinst build
Doxygen Publishing
RST testing
Matplotlib extensions
live
ipython session with ipython directive
dead ipython
inline plots
Syntax Highlighting
math
math equation
math eqnarray
raw html css usage via reST custom roles
table styled with custom role
longtable
figures
tabledoc
Dayabay Sphinx Extensions
Table Doc Directive
DBI Validity Record
DBI Context Query
DBI Validity Lookup Table
Todolist
References
Unrecognized latex commands
¶
None
Indices and tables
¶
Index
Module Index
Search Page
Navigation
index
modules
|
next
|
Offline User Manual 22909 documentation
»