Transfer of Data from Terabyte Disk to BMRC Archives

441 print "Guess: $guess";442 }443 print "\n";444445 if ($links > 0) {446 print "${Prog}: Symbolic Links: file count: $links\n";447 }448449 if ($ozFiles > 0) {450 print "${Prog}: Ignore ’*_OZ.nc’ file count: $ozFiles\n";451 }452453 if ($badFiles > 0) {454 print "${Prog}: Ignore ’.nc.bad’ file count: $badFiles\n";455 }456457 if ($badLinks > 0) {458 print "${Prog}: Ignore ’.nc.bad’ link count: $badLinks\n";459 }460 print "\n";461462 if ($listOut > 0) {463 close LISTOUT;464 print "${Prog}: The file ’" . $listOutput . "’ now contains a list\n";465 print " of the required files with resolved symbolic links.\n";466 print " To use this list with ’afm-ftp.tcl’ it will need to be\n";467 print " pre-processed by the ’path-to-dir-list’ script.\n";468 print "\n";469 }470471 exit 0;472473474 __END__475476 =head1 NAME477478 list-size.pl - Perl script to total up data file transfer sizes.479480481 =head1 SYNOPSIS482483 list-size.pl [options] ls-[l]R-file-list484485 Options:486487 --Man (or ’-M’)488 --help (or ’-h’)489 --list (or ’-l’)490 --size (or ’-s’)491 --verbose (or ’-v’)492493494 =head1 OPTIONS495496 =over 4497498 =item --Man499500 Display the on-line manual page and exit.501502 =item --help503504 Display the help message and exit.505506 =item --list507508 Produces a file called "file-list.txt" containing the paths509 to each of the files in the input with resolved symbolic links.510 To use this list with ’afm-ftp.tcl’ it will need to be511 pre-processed by the ’path-to-dir-list’ script.512513 =item --size514515 Controls the maximum size (of the collection of NetCDF files516 which make up the ’active’ entries in the "file-list.txt" file.517518 =item --verbose519520 Displays the filename of the reference netCDF file list,521 and the number of netCDF files in that list.522523 =back524525526 =head1 DESCRIPTION527528 Reads the "listalldirs.txt" file to get directory path and filenames529 and also file sizes. Next reads the specified (i.e., on the command-line)530 file and totals up the file sizes either from the original ("listalldirs")531 entries, or from the actual "ls -lR" entries if possible.532 Reports the total size of all of the named files.533534 The argument given to the program must correspond to the filename535 of an "ls -lR" (or "ls -R") listing of the target top level directory,536 such as output from the command:537538 ls -lR /export/project/bmrc539540 In some cases the "listalldirs.txt" file contains symbolic link541 entries which indicate that some file sets have been relocated542 on the PCMDI server. This script now attempts to resolve these543 symbolic links and reports of files which remain "unknown", etc.544545546 =head1 FILES547548 listalldirs.txt549 List of all directory and netCDF file names for data transfer550 from the source data filesystem.551552 file-list.txt553 Output file produced when the user specifies the ’-l’ option.554555556 =head1 AUTHOR557558 Lawson Hanson559560561 =head1 BUGS562563 None known.564565 =cut566B.4 Program: ‘path-to-dir-list’The ‘path-to-dir-list’ script reads a specified input-file (i.e., output from ‘list-size.pl’ when runwith its ‘-l’ option) and converts the netCDF file paths it finds there into separate ‘directory:’ and‘filename.nc’ lines, as required by ‘afm-ftp.tcl’, and generates separate files named ‘?-dataN .txt’for each data set where ‘?’ is the file prefix character (which is ‘r’ by default):1 #!/bin/sh2 #3 # Program:4 # path-to-dir-list5 #6 # RCS-Strings:7 # $Source: /bm/gkeep/lih/src/sh/RCS/path-to-dir-list,v $8 # $Revision: 1.6 $9 #10 # Author:11 # Lawson Hanson, 20050510.12 #13 # Purpose:14 # Reads a specified input-file (i.e., output from "list-size.pl -l")15 # and converts the netCDF file paths it finds there into separate16 # "directory:" and "filename.nc" lines, as required by "afm-ftp.tcl",17 # and generates separate files named "?-data{N}.txt" for each data set18 # where ’?’ is the file prefix character (’r’ by default).19 #20 # Updates | By | Description21 # --------+----+------------22 # 20050510| LH | Added code to emit the information to separate "r-data" files.23 # 20050511| LH | Added code to ignore comment lines (’#’) and blank data lines.24 # 20050511| LH | Added code to enable variable file prefix character.25 #26 Prog=‘basename $0 .sh‘2728 fnUsage ( )29 {30 echo31 echo "Usage: ${Prog} [-h] [-p c] [-w output-file ] input-file"32 echo "Where: -h = Display this help/usage message and exit"33 echo " -p c = Specifies the file prefix character (default=’r’)"34 echo " -w file = Specifies an output filename for the whole list"35 echo36 echo "Reads the named input-file (i.e., output from ’list-size.pl -l’)"37 echo "and converts the netCDF file paths it finds there into separate"38 echo "’directory:’ and ’filename.nc’ lines, as required by ’afm-ftp.tcl’"39 echo "and generates separate files named "?-data{N}.txt" for each data set"40 echo "where ’?’ is the file prefix character (’r’ by default)."41 echo42 }4344 pfx="r"45 wFile=""46 wOut=047 noArgs=14849 while [ \( $# -gt 0 \) -a \( $noArgs -gt 0 \) ]50 do51 case $1 in52 -h)53 fnUsage54 exit 055 ;;56 -p)57 pfx="$2"58 shift 259 ;;60 -w)61 wFile="$2"62 shift 263 ;;64 *)65 noArgs=066 ;;67 esac68 done6970 if [ $# -gt 0 ]71 then72 inFile="$1"73 else74 inFile="file-list.txt"18