Friday, 3 March 2017

FME Merge two files that only have some attributes the same


I am trying to join 2 spatial files that have some but not all attribution the same.


One of them (property_template.tab) has no data but has the schema that I want. The other one (cadastre_bpoly.tab) has the spatial info and only some required attributes.



In QGIS a simple merge vector gives the result I need where if an attribute matches it is filled but if not it's left.


In FME I have tried FeatureMerger where the requester/supplier has join on as $(SourceDataset_MAPINFO_5) but it doesn't give a result.


I have also tried ListBasedFeatureMerger.


ALL I need to do is the following:



  1. Get the previous output of the property table and use it as a template (with no data)

  2. Populate it with the cadastre and have it auto fill any attributes that are the same.

  3. Join by attributes with an xls sheet that has a key field but be able to build up the key field using existing columns.


I understand #3 is really a separate question but I included it as it may help in choosing the correct transformers.



For points 1&2 above the merge shouldn't try to do any type of join by attribute but it doesn't allow me to leave the Join Attributes Requester/Supplier blank.


===== UPDATE


Based on Mark Ireland's suggestion enter image description here



Answer



I think - if I understand correctly - that I would do a simple translation for parts 1 and 2. So, in an empty workspace, try following these steps...



  1. Choose Readers > Add Reader from the menubar.

  2. Set MapInfo tab as the reader format and select the Cadastre dataset.

  3. Click OK to add the reader.

  4. Choose Writers > Add Writer from the menubar.


  5. Set MapInfo tab as the writer format and choose a new folder to write to.

  6. Set Table Definition to Import From Dataset and click OK to add the writer.

  7. You will be prompted where to import the schema from. Set MapInfo Tab as the format and select the Property dataset. Click OK to finish adding the writer.


You now have a workspace that will write the Cadastre data to a new MapInfo dataset, using the Property dataset as a form of template.


If any attributes exist in the Cadastre data that also appear on the Property schema, they will get written out. Everything else will be ignored.


There are a few ways to do this, but I think my solution is about as good as you can get. You could use a Dynamic translation, but I don't see any point in that (unless you are going to run this multiple times and the Property dataset will change schema each time). The FeatureMerger/Joiner method mentioned in other answers isn't that simple because it doesn't define the output schema.


The one possible snag is that the Property dataset is empty. FME might not like that. Without features it might think there is no schema and you get an error when adding the writer. The simple solution would be to add a single record to that table. We aren't reading that dataset, just using the structure, so it doesn't matter whether the table is full or empty. That data won't get included, only features from the Cadastre data.


As you say, requirement #3 is really a different thing altogether. But, as everyone else here has mentioned, you can use a FeatureJoiner (or FeatureMerger) transformer to do this. Actually, given that the data is Excel, and you don't really need to read all of it into the workspace, I would use the DatabaseJoiner transformer. So create your key as a new attribute (StringConcatenator or AttributeManager > Text Editor dialog) and use that in the DatabaseJoiner to connect to the Excel sheet.


I found a reasonably suitable demo dataset so I put a short video together, to demonstrate the techniques above. I hope it is of use. You can find it here: https://www.screencast.com/t/tO1F3s3i



No comments:

Post a Comment

arcpy - Changing output name when exporting data driven pages to JPG?

Is there a way to save the output JPG, changing the output file name to the page name, instead of page number? I mean changing the script fo...