I realize I posed this question, and got answers, several days ago (see Field calculator error) , but none seem to really be working, and so am posing again.
Below are two screenshots that show the fields with field properties, and another showing the field calculator and the equation with the "answer" field populated by 0s. I apologize for them not being next to each other or in line with the appropriate text, but the web formatting keeps overlaying them.
Here is an overview of the problem: I am trying to do a in field calculator, where I'm trying to populate a new field with the values from a fairly simple equation of one field divided by another (Cnt1 by Pop1 in this case).
Both fields that I am dividing are double, and the value, while small is not infinitesimal, approx 0.01765.
I have tried creating various types of new fields: double, long, float, and played around with the scale/precision/properties, but everytime it returns "0" values. I know the values will be very small, i.e .04 but it doesn't seem to be a decimal display problem as I tried running it with Pop1/.4 and it correctly returned a decimal.
I've tested dividing the Cnt1 by .5 and that gives an appropriate decimal value, so it's not the new field format; the one thing that seems really strange is when I add (in field calculator python version): !Cnt1!+!Pop1! (which is 38+218699) I get 2.187, however, I suppose this could be related to the field being classified as float with precision=4, scale=3?
I've also tried the suggestions of @blord-castillo of doing:
1.0* (Cnt1/Pop1)
and also 1.0* (int(Cnt1/Pop1)) But neither works, I'm at my wits end on this one, and would greatly appreciate any help!
Answer
I just ran a test at ArcGIS 10.0 SP5.
You can see my results below.
To achieve this in a file geodatabase polygon feature class, I had field cnt1 as Short Integer, pop1 as Long Integer, and Stnd3 as Double.
In the Field Calculator I used:
Stnd3=
float(!cnt1!) / float(!pop1!)
I then used the context menu of the Stnd3 field to access the Numeric... button on its Properties to set the Number of decimal places to 9.
Is this the result you are after?
No comments:
Post a Comment