grass - Delineating large scale watershed?

I would like to do some watershed delineation at national/regional level (using SRTM-90 data).

I have experimented with r.watershed (GRASS 6.4.3) but it fell over when I tried to do big stuff.

I have tried r.terraflow (GRASS 6.4.3) but don't really know what I am doing with this... Seems quite old so maybe there is something better these days?

No luck with ArcGIS at all for big stuff (but available to me if somebody has a solution using this software).

Can anybody recommend practical tools for big watershed delineation projects?

Answer

I have a tremendous respect for GRASS and the r.tarraflow algorithm and I'm sure that given enough effort, you would be able to make it work for this application. But as an alternative, I develop a cross-platform free and open-source GIS called Whitebox Geospatial Analysis Tools (download here). Here is an example for how to use it for hydrological processing of massive DEM data. Using the Retrieve SRTM Data tool, I was able to download all of the SRTM 3-arcsecond tiles for the British Isles, mosaic them into a single 13,201 x 14,401 726 Mb DEM in about 10 minutes.

I then used the Breach Depressions (Fast) tool on the DEM, which performed hydrological correction, calculation of the D8 flow direction raster, and the flow accumulation step. It took about 8 minutes to perform that analysis on my Macbook Pro (2.8 GHz, 16GB RAM).

I performed a simple thresholding to extract major British streams (and vectorized them for display purposes:

Here are the watersheds for each outlet in the stream networks layer:

From starting the SRTM download process to the final watershed map took me about 35 minutes of processing. It probably would have taken less time but I had to make dinner in there too ;) As with everything, your milage will vary depending on your system. When you're processing very large rasters, it's a good idea to have 16-64GB of RAM and an SSD is also a major plus. For further details, this link describes process of extracting watersheds from DEMs using Whitebox GAT. Granted, the British Isles aren't a particularly large landmass; if I had more time I would have liked to have done Brazil just for fun!

EDIT

washmaps mentioned in their comment above that they were interested in extracting the Nile River Basin. So I thought that I would experiment to see if I could perform this analysis. Downloading the 640 SRTM 3-arcsecond tiles took 19 minutes using the Retrieve SRTM data tool in Whitebox and another 18 minutes to create a seamless mosaic consisting of 38,401 rows by 22,801 columns, which is a little over 875,000,000 pixels. The DEM is 3.26 GB in size. Filling the NoData holes took 4 minutes. I then performed hydrological correction using the Breach Depressions (Fast) tool, which took 49 minutes 8 seconds. I realized after the fact that I should have run a depression filling algorithm after the initial breaching as well, but it was too late. Calculating the D8 flow direction grid took 3 minutes, and the subsequent flow accumulation operation took 4 minutes 14 seconds. Finally, extracting the watershed for a point on the Nile delta took 2 hours and 9 minutes. Again, all of this was using my 13 inch Macbook Pro 2.8 GHz dual core with 16 GB RAM and an SSD.

And of course, as soon as I saw the watershed (brown polygon on map), I realized that I should have downloaded data further south of Lake Victoria and west in Sudan (that border is obviously the Nile basin). Oh well, as a proof of concept, using fairly standard equipment, this demonstrates that Whitebox GAT can be used to delineate large-scale watersheds.

UPDATE

Simply because I can never leave things partially completed, I decided to use my workstation and get the job done right.

The watershed in (very nearly) complete and the stream network is the DEM-extracted network. Here is a long profile for the Nile (although I should have converted distance to m):