Other aop/env

Table Of Contents

Previous topic

DbiMonitor package : cron invoked nosetests

Next topic

Dybinst : Dayabay Offline Software Installer

This Page

Daya Bay Links

Content Skeleton

Env Repository : Admin Infrastructure Sources

Most of the development history and documentation (commits, tickets, wikipages) of the admin infrastructure is managed in the env SVN/Trac instance at NTU env:wiki:WikiStart . This includes:

  1. scripts for backup/recovery of the MySQL DB offline_db
  2. scripts for backup/recovery of the SVN repository and Trac instance
  3. patches against Trac and various Trac plugins such as bitten

Using the Trac tags is the best way to locate this documentation, for example:

  • env:tag:SCM provides a list of Source Code Management related pages

Installing Env

Follow the instructions on the front page env:wiki:WikiStart to install env, starting from:

cd $HOME ; svn checkout http://dayabay.phys.ntu.edu.tw/repos/env/trunk/ env
cd $HOME/env ; svn update

Hook up env to the bash shell of the user by adding the below to the .bash_profile:

export ENV_HOME=$HOME/env
env-(){  . $ENV_HOME/env.bash  && env-env $* ; }
env-     ## precursor function

Heirarchy of Bash Functions

The env- precursor function defines other precursor functions such as

  1. scm-
  2. scm-backup-
  3. db-

These precursor functions are very simple, all following the form of defining a set of related functions and setting up any environmenrt with eg scm-env:

scm-(){         . $(env-home)/scm/scm.bash && scm-env $* ; }
scm-backup-(){  . $(env-home)/scm/scm-backup.bash && scm-backup-env $* ; }

Certain conventions are followed:

  1. precursor function names end with a hyphen: -
  2. functions defined by precursor functions like scm- are named to extend that name, ie scm-create and scm-wipe
  3. certain standard functions are included for all eg scm-vi, scm-backup-vi, db-vi all open the functions source in the vi editor
  4. other standard endings include *-usage, *-source etc

Conventional naming structure provides convenient tab completion:

[blyth@belle7 ~]$ scm-
[blyth@belle7 ~]$ scm-<TAB>
scm-                 scm-create           scm-env              scm-postcommit-      scm-rename           scm-src              scm-vi
scm-backup-          scm-eggcache         scm-postcommit       scm-postcommit-test  scm-source           scm-usage            scm-wipe

The env- function defines some aliases such as t which provides interactive access to function definitions from the commandline:

[blyth@belle7 ~]$ t t
t is aliased to `type'
[blyth@belle7 ~]$ t scm-create
scm-create is a function
scm-create ()
{
    local msg="=== $FUNCNAME :";
    local name=$1;
    shift;
    [ -z "$name" ] && echo $msg an instance name must be provided && return 1;
    svn-;
    svn-create $name $*;
    trac-;
    trac-create $name
}
[blyth@belle7 ~]$ t scm-vi
scm-vi is a function
scm-vi ()
{
    vim $(scm-source)
}
[blyth@belle7 ~]$ scm-source
/data1/env/local/env/home/scm/scm.bash
[blyth@belle7 ~]$ scm-vi

As far as possible the functions seek to abstract away node specific details, such as directory paths and basis application layout (eg different versions of apache have files in different places). The functions should shield the user from these node specifics.

Warning

the functions are used on many different nodes, this requires care to avoid breaking things for other nodes by ignoring the node agnostic approach

Node Characterisation

Node abstraction is achieved by node detection and the setting of standard envvars such as NODE_TAG by the elocal- precursor:

[blyth@belle7 home]$ t elocal-
elocal- is a function
elocal- ()
{
    . $(env-home)/base/local.bash && local-env $*
}
[blyth@belle7 home]$ t local-env
local-env is a function
local-env ()
{
    local dbg=${1:-0};
    local msg="=== $FUNCNAME :";
    [ "$dbg" == "1" ] && echo $msg;
    export SOURCE_NODE="g4pb";
    export SOURCE_TAG="G";
    export LOCAL_ARCH=$(uname);
    export LOCAL_NODE=$(local-node);
    export NODE_TAG=$(local-nodetag);
    export BACKUP_TAG=$(local-backup-tag);
    export SUDO=$(local-sudo);
    export SYSTEM_BASE=$(local-system-base);
    export LOCAL_BASE=$(local-base);
    export ENV_PREFIX=$(local-prefix);
    export VAR_BASE=$(local-var-base);
    export SCM_FOLD=$(local-scm-fold);
    export VAR_BASE_BACKUP=$(local-var-base $BACKUP_TAG);
    export USER_BASE=$(local-user-base);
    export OUTPUT_BASE=$(local-output-base);
    local-userprefs
}

Note

simple usage of echo from bash functions to return values to other functions, requires care regards extraneous output

The NODE_TAG is very widely used for branching on node specifics:

[blyth@belle7 home]$ local-nodetag
N

For example the local-scm-fold emits the path used as the base for backups:

[blyth@belle7 home]$ local-scm-fold
/var/scm

[blyth@belle7 home]$ t local-scm-fold
local-scm-fold is a function
local-scm-fold ()
{
    case ${1:-$NODE_TAG} in
        WW)
            echo /home/scm
        ;;
        *)
            echo $(local-var-base $*)/scm
        ;;
    esac
}
[blyth@belle7 home]$ t local-var-base
local-var-base is a function
local-var-base ()
{
    local t=${1:-$NODE_TAG};
    case $t in
        U)
            echo /var
        ;;
        P)
            echo /disk/d3/var
        ;;
        G1)
            echo /disk/d3/var
        ;;
...etc...

A case function is used on the NODE_TAG to locate the different places on