Advanced GIS Class Spring 2007 Batch Processing of Radar Reflectivity Files for Value Extraction Newfel Mazari Under the direction of Dr Hongjie Xie University of Texas at San Antonio Abstract Some GIS tools are easy to use for simple and unique tasks but for redundant work such as extracting attribute tables form a large set of GIS format files I need to use more sophisticated approaches geoprocessing comes very handy to do this kind of work especially for batch processing a series of shape files to extract attribute tables to an ASCII output all tables have to be merged together while keeping the file names associated with each output table For my thesis research the problem reside on how to extract reflectivity values form radar shape files that correspond to a point location in my case its my rain gage location The output of the batch processing is very promising but I still have not fix the error handling part of the script In fact the whole script crashes if the intersection of the radar file with the rain gage file is null the intersection is processed but the export XY and filed attribute generates the error because there is no XY coordinates for the intersection and no attribute fields My way around this problem is to eliminate manually the bad files and restart the script again I also noticed that the script does not tolerate to reprocess the same files again stored in the workspace folder so I had to erase them also Introduction The use of radar products for Iather forecasting hydrological modeling or flood prediction and management is widely used because its low cost and spatial coverage but many problems still persists with radar products validation and comparison to other ground surface meteorological stations Precipitation product is the most used nowadays for forecasting and water resmyces management several product exists for precipitation estimates they are usually named product levels or stages the most basic one is radar base reflectivity product 19 stage II Reflectivity Z is the backscattered radar poIr radar scans the atmosphere at regular angles and intervals of time two modes are used by the NWS National Iather Service Bureau 1 clear sky mode 2 precipitation mode Radar scans the entire range 230Km radius in 5 to 15 minutes 6 minutes scan are used for precipitation mode and 5 minutes for severe storm conditions The reflectivity is converted through a poIr formula to rain rate R Z a Rb the parameters a and b are determined by the specific climate conditions for each region and by Iather conditions different precipitation types may have different optimum parameters Nevertheless radar rainfall estimate is a volume scan that is compared to ground surface rain gauges which creates discrepancies betIen the two measurements radar and rain gauge other problems due to radar physics hardware and calibration limitations adds more to the error induced in radar rainfall estimates plus atmospheric conditions can increase the radar bias such as hail contamination and the virga effect The present class project is part of a research project for radar reflectivity validation and comparison to a network of rain gauges installed in a single radar cell data processing and methods used to extract the reflectivity form radar files are of primary concern to my project I will introduce in this brief paper a GIS based method for batch processing radar reflectivity files for later analysis and comparison with rain gauge precipitation estimates Data For my thesis research the problem reside on how to extract reflectivity values form radar shape files that correspond to a point location in my case its my rain gage location Radar reflectivity files come in different formats Vector format 1 Shape file 2 Ill Known Text Raster format 1 Geotiff 2 ESRI ASCII Grid 3 ESRI Binary Grid 4 GrADS Binary Grid 5 HDF 6 NetCDF Only shape file and Gird formats can be used in a GIS application at first I selected the grid format to extract the reflectivity values for my rain gage location but the results Ire very poor this due to three problems 1 Grid uses resampling procedure to build the raster 2 Cels are identical in size 0 980 0 980Km 3 Each cell value is a neighborhood average My data consists of over 1500 shape files stored in a disk the files Ire downloaded form the NWS radar inventory products then exported as shape file to the computer hard drive another shape file for the rain gage is used and stored in a different subfolder which will be used as basis for the reflectivity value extraction The location of the rain gage will determine which radar cell value will be extracted figure 1 a b shows the rain gage location and the radar cells Methods ArcGis provides several useful tools to manipulate shape files the Analysis Tool box is the most suited for this file format I will use a second tool form the Statistic toolbox to extract the cell values to and text file output ASCII file My first concern is to locate the cells form each file that contains completely the rain gage the solution is used the Select layer by Location Tool figure 2 But a the problem with this tool is that each shape file has to be loaded to Arc Map as a Map layer which constitutes a big problem when I are dealing with hundreds of large files so I have to find something that does nit require loading each file as a layer the second problem is to create a layer form the selection then export its attribute table to an output file so this tool is useful for a limited files number because it works manually Another option is to use the Intersect tool Overlay Analysis Tool box to force the intersection of a polygon with a point and join their attribute tables the output of the intersection is set to be a point geometry to limit the usage of disk space the intersection files will be stored in a different subfolder for later use Figure1 a b Location of rain gauges and study area The extraction process is done by a tool in Arc Map called Export feature Attribute to ASCII this tool is script based and stored in the standard toolbox it exports the X and Y coordinates of the feature plus one choice of field attributes for my case it will be the value Field The simplest way to start a geoprocessing session is to use the Model Builder in ArcGis Figure 3 test the model functionality set all the necessary parameters save it as a Model then export the model to a script ArcGis gives three options for scripting languages visual Basic Jython and