Tuesday, 11 September 2018

pyqgis - Reading/writing utf-8 encoded data using Fiona?



So I have a 'cpg' file with my 'in_file' shapefile data (definitely set to 'utf-8'). I was using 'meta' to create the same schema/crs as 'in_file'...though it runs, my text fields are borked on anything that isn't ascii (note the cjk - 'ZH_CN' - is coming back as codepoints).


ex:


    {'geometry': {'type': 'MultiPoint', 'coordinates':
[(13531245.475704141, 2886003.2689278126)]}, 'type': 'Feature', 'id':
'0', 'properties': OrderedDict([(u'EN_US', u'Taipei City'), (u'ZH_CN',
u'\u53f0\u5317\u5e02'), (u'ID', 1668338.0), (u'DE_DE', u'Taipei
City'), (u'ES_ES', u'Taipei City'), (u'FR_FR', u'Taipei City'),
(u'JA_JP', u'Taipei City'), (u'KO_KR', u'Taipei City'), (u'PT_BR',
u'Taipei City')])}


My first thought was to change:


with fiona.open(in_file, 'r') as input:

to:


with fiona.open(in_file, 'r', encoding='utf-8') as input:

...but that resulted in:


UnicodeEncodeError: 'latin-1' codec can't encode characters in
position 0-2: ordinal not in range(256)


Reading it without the encoding seems to be the start of my problem...however I was surprised that the error message arose when I did the above.


def process_file(self, in_file, out_file, compare_file, dist):
# open input file and compare file
# loop over each
with fiona.open(in_file, 'r') as input:
meta = input.meta
# The outFile has the same crs, schema as inFile
with fiona.open(out_file, 'w', **meta) as output:
with fiona.open(compare_file, 'r') as compare:
# Read shapely geometries from file

# Loop through all shapely objects
# type(item) = 'dict'
for item in input:
geom = item['geometry']
my_shape = shape(geom)

# check if it is a multipoint or point
if geom['type'] == 'MultiPoint':
# break Multipoints into points
# i.e.


for pt in my_shape:
single_point = self.move_point(pt, compare, dist)
mpoint = MultiPoint([(single_point.x, single_point.y)])
mpoint_for_merge = shape(mapping(mpoint))
# write to outfile - WHERE EVERYTHING NOW WORKS
output.write({'geometry':mapping(mpoint_for_merge), 'properties': item['properties']})

elif geom['type'] == 'Point':
# return of move_point is a shapely geom

# i.e.
my_shape = self.move_point(my_shape, compare, dist)
# write to outfile
output.write({'geometry':mapping(my_shape), 'properties': item['properties']})

else:
raise ValueError('unhandled geometry type: ' + repr(geom.type))


No comments:

Post a Comment

arcpy - Changing output name when exporting data driven pages to JPG?

Is there a way to save the output JPG, changing the output file name to the page name, instead of page number? I mean changing the script fo...