Personal tools
You are here: Home → Project Documentation → Getting Started Guide → iVec Infrastructure Overview
« February 2012 »
February
MoTuWeThFrSaSu
12345
6789101112
13141516171819
20212223242526
272829
 

iVec Infrastructure Overview

A brief introduction to iVec infrastructure as related to the WASTAC project

Introduction

  • iVec provide a range of facilities for storage, computation and batch processing
  • WASTAC have added their own equipment for database and portal requirements

Processing

Cognac

  • cognac.ivec.org
  • The main processing facility
  • 170 Itanium CPUs in a shared memory model
  • 1TB of disk space for medium term storage (/projects)
  • 10TB of short term storage (/short)
  • Login interactively 
  • Submit jobs to a batch queue using PBS
  • Runs SUSE Enterprise Linux

Storage

The PB-Store

  • pbstore.ivec.org
  • 1-PB of heirachical storage (/pbstore)
  • Appears as one big disk
  • Files are either on-line, on-tape ("offline"),  or on-line and on-tape ("online")
  • Files are stored on two tapes - in case one tape fails
  • Off-line files come on-line on-demand (when you open them)

Storage

PB-Store (Continued)

  • pbstore.ivec.org runs Solaris x86 and is the main access point to the PB file system
  • The file system is actually implemented by a range of servers (pb-sh-01, pb-sh-02) and tape robots
  • pbstore.ivec.org has high-speed access to the file system (fibre-channel)
  • cognac does not have direct access to the PB-Store
  • wastac.ivec.org has read-only NFS access

Database and Portal

wastac.ivec.org

  • installed specifically for the WASTAC project to provide a database and web-portal facility
  • Solaris x86, 2 x 4-core AMD, 24Gb RAM, 16 x 136Gb SAS running in a RAID arrangement
  • hosts...
    • plone - a web application and content platform
    • subversion - software repository
    • postgres - a relational database with spatial extensions
  • has a read-only NFS mount of the pbstore (/pbstore)

Caveats

  • cognac has very limited storage
  • cognac does not have access to the pbstore - data needs to be copied in and copied out (and deleted) as required
  • cognac has a 1Gb ethernet link - can easily become a bottleneck when transfering data to/from pbstore

Cognac Storage Detail

cognac$ df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/md3              123G   82G   42G  67% /
udev                  184G  3.1M  184G   1% /dev
/dev/md0              191M   33M  158M  18% /boot/efi
/dev/md4              123G  123G  528M 100% /opt
/dev/lxvm/Jobfs       934G   36G  898G   4% /jobfs
/dev/lxvm/Projects    977G  946G   32G  97% /projects
/dev/lxvm/Short       9.9T  8.8T  1.1T  90% /short
/dev/lxvm/Home        391G  133G  258G  34% /home

Pbstore Storage Detail

pbstore

pbstore% df -h | egrep 'File|export'
Filesystem             size   used  avail capacity  Mounted on
samqfs1                 71T    66T   5.2T    93%    /export/samqfs1
samqfs2                 16T    36M    16T     1%    /export/samqfs2
utility                500G   9.6G   490G     2%    /export/utility

Pbstore Storage Detail (Continued)

pbstore

pbstore% df -h /pbstore/wastac /pbstore/remote
Filesystem             size   used  avail capacity  Mounted on
samqfs1                 71T    66T   5.1T    93%    /export/samqfs1

Adminstration

  • Accounts on all systems are managed by iVec
  • iVec provide all system adminstration for all systems
  • wastac.ivec.org is an exception - sys admin by us, but iVec can manage it also
  • help@ivec.org

Cognac (General Software)

  • a large amount of processing software and compilers are installed on cognac
  • with a large number of users and software, conflicts in names and library versions are inevitable
  • a potential nightmare to manage

Cognac (General Software)

GNUmodule

  • iVec use something called GNUmodule to manage installed software package and prevent conflicts
  • GNUmodule allows you to load only the package(s) that you require into your environment
  • behind the scenes GNUmodule updates your PATH, LD_LIBRARY_PATH and other environment variables as you load/unload modules

Cognac GNUmodule

  • useful commands
    • module avail - list all software packages available for use
    • module load package - load the named package
    • module unload package

Cognac WASTAC Software

  • iVec will install software for you if you request them to

  • If you're experimenting / getting familiar, you may be better off installing it somewhere yourself to get started 

  • See Mark for advice about where to install software if you choose to do this

Cognac WASTAC Software (Continued)

/opt/wastac

  • The WASTAC iVec project (Nick and I) has been amasing our software under /opt/wastac
  • /opt/wastac/tools contains SeaDAS, IMAPP, MODISL1DB, CAPS and some other tools
  • /opt/wastac/local/modulefiles contains GNUmodule package-files for the software - so you can load them as needed
  • Your colleagues may have more optimized or improved versions
 
Document Actions
Log in


Forgot your password?