qgis - How to invert polygons (or other options for exterior styling)?

I have a shapefile containing many polygons that I want to use to mask a raster. I have tried gdalwarp -cutline -crop_to_cutline -srcnodata -9999 -dstnodata -9999 , but the results are visually unappealing because of the low resolution of the raster (i.e. jagged edges). It would look better if I could simply place the vector layer over the raster as a masking layer in the map. I would need to see the underlying raster inside the polygons, but not outside. Two options have occurred to me for accomplishing this.

1. Invert the polygons and save as new shapefile then fill with opaque background color.



2. Style the vector layer such that inside the polygons is transparent and outside is opaque.

Being new to GIS, I haven't figured out how to accomplish either of these approaches. I have at my disposal the gdal framework, QGIS, and GRASS. I think that I would prefer a GRASS command-line solution, as I may need to repeat this process more than a few thousand times in the next few days. Also, please feel free to direct me to the appropriate documentation that I have obviously missed.

I hope that this question isn't a duplicate of one of the many I've read about masking rasters with vectors. It appears to me that this case is different because I specifically want the final map to display partial pixels from the raster via a smooth cut by a vector.

Answer

You could achieve this using GRASS GIS and the r.mask module by specifying the -i flag to create the inverted MASK. If you need to repeat this for several Shapefiles, you could probably automate this process with a shell script something along the lines of:

for file in `ls *.shp`
do
   v.in.ogr dsn=$file output=${file%.shp}
   v.to.rast input=${file%.shp} output=${file%.shp}_r use=cat

   r.mask -i ${file%.shp}_r 
   g.copy rast=MASK,${file%.shp}_mask
done

You may want to change the resolution of the location before converting the vector to a raster, e.g. g.region res=...