Sunday 21 October 2018

pyqgis - Is it possible to use memory layers in QGIS processing algorithms?


I am building a QGIS plugin that connects to a MySQL database in the local network, and then adds a subset of one of the tables to an in-memory layer; the subset is based on data currency (only taking the most recent observation for each location where measurements are made). This memory layer is successfully created.


However I then want to run some geoprocessing algorithms, and I'm having trouble using the in-memory layer in any of them.


    self.stationuri = "point?crs=epsg:4326&field=id:integer&field={}:double&index=yes".format(self.cb_field.currentText())
self.vlayer = QgsVectorLayer(self.stationuri,"scratch","memory")
if not self.vlayer.isValid():
raise Exception("Failed to create in-memory layer")
self.vlayer.startEditing()

for i,r in enumerate(result): # Result is row-by-row result of SQL query
# Add features
...
self.vlayer.commitChanges()
self.vlayer.updateExtents()
# Add layer to map
QgsMapLayerRegistry.instance().addMapLayer(self.vlayer)
# Layer is successfully added to map with all features and geometry
# BELOW IS WHERE IT FALLS APART
try:

processing.runandload("gdalogr:gridinvdist",self.vlayer,self.cb_field.currentText(),2,0,0,0,0,0,0,0,'Float32',None) # None = in-memory output; I get the same error if I specify a string path and filename.
except Exception, e:
raise e

No exception is raised, but no output is produced or added to the TOC, but the following log is made in processing.log:


INFO|Mon May 04 2015 11:28:23|GDAL execution console output|/bin/sh: 1: /tmp/processing/bbebe7599c83446d9c2b03a251879657/OUTPUT.tif: not found|/bin/sh: 1: -zfield: not found||FAILURE: Source datasource is not specified.|Usage: gdal_grid [--help-general] [--formats]|    [-ot {Byte/Int16/UInt16/UInt32/Int32/Float32/Float64/|          CInt16/CInt32/CFloat32/CFloat64}]|    [-of format] [-co "NAME=VALUE"]|    [-zfield field_name] [-z_increase increase_value] [-z_multiply multiply_value]|    [-a_srs srs_def] [-spat xmin ymin xmax ymax]|    [-clipsrc |WKT|datasource|spat_extent]|    [-clipsrcsql sql_statement] [-clipsrclayer layer]|    [-clipsrcwhere expression]|    [-l layername]* [-where expression] [-sql select_statement]|    [-txe xmin xmax] [-tye ymin ymax] [-outsize xsize ysize]|    [-a algorithm[:parameter1=value1]*]    [-q]|     ||Available algorithms and parameters with their's defaults:|    Inverse distance to a power (default)|        invdist:power=2.0:smoothing=0.0:radius1=0.0:radius2=0.0:angle=0.0:max_points=0:min_points=0:nodata=0.0|    Moving average|        average:radius1=0.0:radius2=0.0:angle=0.0:min_points=0:nodata=0.0|    Nearest neighbor|        nearest:radius1=0.0:radius2=0.0:angle=0.0:nodata=0.0|    Various data metrics|        :radius1=0.0:radius2=0.0:angle=0.0:min_points=0:nodata=0.0|        possible metrics are:|            minimum|            maximum|            range|            count|            average_distance|            average_distance_pts|

The important part seems to be FAILURE: Source datasource is not specified. However self.vlayer.isValid() == True, so I don't see what is wrong with my input. I have tried substituting self.vlayer with 'memory:scratch' in the call to processing.runandload, but then I get the following error printed to the console (but not raised): Error: Wrong parameter value: memory:scratch.


I get the same issue when running this through the QGIS GUI, and using the dropdown menu to select my scratch layer which is in the TOC. This occurs whether I specify the output raster as in-memory or specify a location on disk.


This question seems similar, but their solution was to add the memory layer to the TOC before using it. I am already doing that and yet the error persists.



I thought that this was a general issue with memory layers and the QGIS geoprocessing algorithms, but the following works without issue:


processing.runandload("qgis:fixeddistancebuffer",self.vlayer, 500, 5, True, "output_buffer.shp")

What am I doing wrong? Why can't my memory source dataset "be specified" in some of the processing algorithms?


EDIT: here's the source code of gdalogr:gridinvdist if that's useful.



Answer



It seems like memory layers cannot be used as input for GDAL/OGR processing scripts because Processing fails to properly prepare the data for use with ogr2ogr. That's why, for example, the QGIS buffer tool works but the GDAL/OGR buffer tool fails:


Algorithm Buffer vectors starting...
GDAL command:
cmd.exe /C ogr2ogr.exe "C:\Users\anita\AppData\Local\Temp\processing70e5e0852cb9456ba2e3780f8386122e\86d237c8f41443f58a230a8133172047\OUTPUTLAYER.shp" point?crs=EPSG:4326&memoryid={6772bccd-f55d-461d-aff6-6271ded02eea} point?crs=EPSG:4326&memoryid={6772bccd-f55d-461d-aff6-6271ded02eea} -dialect sqlite -sql "SELECT ST_Buffer( geometry , 1000 ),* FROM 'point?crs=EPSG:4326&memoryid={6772bccd-f55d-461d-aff6-6271ded02eea}' "

GDAL command output:
FAILURE:
Unable to open datasource `point?crs=EPSG:4326' with the following drivers.
-> JP2ECW
-> OCI
-> SOSI
...

Processing would have to somehow prepare the data (save it to a file) and then feed it to the GDAL/OGR tool.


I opened a ticket: Cannot use memory layers with OGR tools



No comments:

Post a Comment

arcpy - Changing output name when exporting data driven pages to JPG?

Is there a way to save the output JPG, changing the output file name to the page name, instead of page number? I mean changing the script fo...