I have a polygon featureclass containing an integer field 'gridcode'. I've added 2 new float fields VB_calc and Python_calc, both of which should be equal to gridcode / 1,000,000.
If I calculate the field using the Field Calculator with the VB Script [fieldname] syntax, the result is as expected.
If I use the Python !fieldname! syntax, the results are all zero.
What am I doing wrong here? I entered those expressions by double-clicking the gridcode fieldname in the calculator. (My goal is to use this within a stand-alone Python script but I'm debugging it here since the script isn't working either.) Both fields are definitely of type FLOAT.
Answer
As you've discovered python truncates integers that are divided - this may include floating point fields with integers stored in them, sometimes 'duck typing' isn't your friend!
To force the issue and produce a floating point answer from a division float your number first:
float(!GRIDCODE!)/100000.0
It is best to do this always when dividing otherwise if your enumerator (value in the field) is an integer you could get a truncated answer thanks to pythons' sometimes handy duck typing of variables.
For example if most of your values are like 999.999999 but one is actually 1000 and you divide by 10000 you will get around 0.99999999999 as the answer for most and 0 for the one that has an integer value...
No comments:
Post a Comment