Personal tools
You are here: Home Project Documentation Meetings Status Update for Wastac GM 22/08/07
Document Actions

Status Update for Wastac GM 22/08/07

Summary of achieved items in last 3 months.

What
When 2007-08-22
from 00:00 to 00:00
Where Leeuwin Centre
Attendees Merv Lynch
Add event to calendar vCal
iCal

AVHRR Processing

  • Built and deployed latest CAPS software.  Many initial problems but now functioning well to produce L1B.
  • Wastac method of CAPS utilization is no longer supported.  AMC (orchestration layer over CAPS) now deployed for community use.
  • Submitted unsatisfied metadata requirements for AMC to Edward King & Peter Turner.  Received and tested new AMC software for level 1B HDF generation, including standard CLAVR cloud mask.
  • AAPP is also deployed and L1B generation verified with UK Met Office.  No HDF output currenrtly, so a long-term backup option.
  • All fully documented.


MODIS Processing

  • MOD35 is being used for cloud detection - thanks to Florian Goessmann for getting this algorithm working.


iVec Infrastructure


Portal and Database Server
  • x86 server provided by iVec for database and portal function hit memory and IO limitations which halted progress on this tier for some time. This was expected considering it was relatively low specification.
  • iVec stated that they had no short to medium term plans to provide database or portal infrastructure under the iVec initiative but would house hardware we provide.
  • Presented an overview of the WASTAC database project to Sun Microsystems who then agreed to provide hardware for our database and portal requirements. Sun subsequently delivered a V880 (4 x uSPARC-III, 28 Gb) server to iVec for our requirement.
  • WASTAC portal software stack (portal, code management, database) rebuilt & migrated to new server; and verified to meet our performance requirements.
Storage
  • iVec's tape storage system hit maximum capacity and is since only partly operational. It is possible to access existing on-disk data but not to write additional data nor recover data on tape. This impeds our ability to progress processing work to some extent but is not a complete block.
  • iVec have procured their replacement petabyte storage system from Sun and are in the process of comissioning it. This appears to be getting delayed no ETA yet.

Data Gridding and Ingest

  • Have loaded 200 MODIS scenes into the gridded database.
  • Statistics on this dataset indicate that the data volume for 10x10km tiles is higher than expected and too high for practical purposes (approximately 100,000 "tiles" per scene - or 7Gb SQL data for 200 scenes).
  • We are consequently investigating various means of reducing the data volume and believe we can acheive this.

Web Search Portal

  • Master swath listing now viewable through wastac.ivec.org
  • Search interface just around the corner.
  • Continued comprehensive documentation of all project activities.  Significant ranking in Google searches of MODIS and AVHRR processing.