Make a Qgis layer update from a changed data source

I'm trying to get layers to update automatically when their data source changes. I'm using R to write a shapefile with an attribute, and colouring according to that attribute in QGIS.

I want to write a new shapefile with different attribute values, and have the Qgis map colours update. Step 1 is triggering that process, step 2 is making the layer reload from the modified shapefile. Its step 2 I am worrying about here.

Other questions/mailing list chatter mentions using triggerRepaint on the layer - that doesn't work. Other suggestions include setCacheImage(None) and again that doesn't work. The layer does update eventually, but I really can't see the logic, and sometimes it happens by surprise after I've done nothing. Or maybe I did something two minutes ago.

The one reproducible way of getting it to update is to duplicate the layer from the legend menu - the duplicate always gets its data from the current shapefile, and the original layer updates itself too! So there must be some way of doing it.

I think it was working better in 2.8, but this is 2.10 so maybe there's a new bug somewhere.

Related, but doesn't work for me in 2.10:

How to automatically reload raster layers if source is changed in QGIS?

Other things I've tried:

• layer.dataProvider().dataChanged.emit() - worked once, then not again on the same layer

I think I've tracked down why duplicating the layer works - if I create a new throwaway layer based on the updated layer and then call .triggerRepaint() on the updated layer, it updates on the map canvas:

QgsVectorLayer( layer.source(), "layer copy", layer.providerType() )
layer.triggerRepaint()

If I use a different layer source it doesn't work, so it seems to be if you create a layer object based on the same layer source...

A quick test just now with a raster layer (from a GeoTIFF), and just calling rlayer.triggerRepaint() seems to reliably update the view of the raster in the map canvas.

Answer

This is related to the introduction of the OGR connection pool. [1]

Before QGIS 2.10, a file was reopened on every single access (e.g. repaint).

Since QGIS 2.10 the file handle is kept open and this means if a file is replaced the handle still points to the old file on Unix based systems.

QGIS 2.10: workaround

Unfortunately there is no API to nicely force QGIS to reopen the file in QGIS 2.10. As a workaround you can use an ugly hack:

layer.dataProvider().changeAttributeValues( { -1: { 0: 0 } } )
layer.triggerRepaint()

QGIS 2.12: solution

I just introduced a new method which will be available starting from QGIS 2.12:

layer.dataProvider().forceReload()

layer.triggerRepaint()

General approach

If you have the possibility to control how the file is being overwritten you can open the existing files with write permissions and change the content instead of replacing the files completely (delete/recreate) on disk.

[1] The connection pool was introduced to significantly speed up access to certain data sources.