Is it possible to have self intersecting polygons in shapefile?

GIS vector data for example shapefile contains polygonal data.

Lets assume, we have polygon A which has self-intersection which means that the edges in this polygon intersect with each other.

Lets say if we want to intersect A with polygon B, then the steps of computing the overlaid polygon differs whether A is a polygon with self-intersections or not.

Do the polygons in one map layer have self-intersections or not?

I searched for this question on internet and I found couple of instances where people complain about error caused while importing data due to the presence of self-intersecting polygons.

Also I got the impression that self intersecting polygons are not possible in certain file formats.

Answer

Here are relevant excerpts from pages 8 and 9 of the Shapefile Technical Description (emphasis mine):

A polygon consists of one or more rings. A ring is a connected sequence of four or more points that form a closed, non-self-intersecting loop.

And:

• 
  

  Polygons stored in a shapefile must be clean. A clean polygon is one that

  
  
  
  
  1. Has no self-intersections. This means that a segment belonging to one ring may not intersect a segment belonging to another ring. The rings of a polygon can touch each other at vertices but not along segments. Colinear segments are considered intersecting.
  
  
  
  

So, as far as the shapefile specification is concerned, the answer is no, polygons may not intersect themselves.

This is a conceptual rule, not a limitation of the file format. It is quite possible to construct shapefiles containing self-intersecting polygons. However, since self-intersecting geometry is invalid according to the specification, applications may differ in how they handle such cases: some may attempt to clean up the features, while others may [justifiably, if less helpfully] consider the shapefile malformed. Most problematic may be cases where the software assumes a shapefile conforms to the specification (ie, contains no self-intersections) but does not validate it before performing geometric operations, which may yield wrong results.

It's unclear exactly what you want to do, but if you're generating shapefiles, I'd advise against outputting polygons with self-intersections, as it may cause difficulty for your data users. Likewise, if you're writing some sort of algorithm to process shapefiles, you do not need to support self-intersecting polygons to comply with the specification - but if possible, I'd advise you check the input files are not malformed before proceeding.