I'm not aware of any way to do this in a .geo file, under the GMSH application, or using the libgmsh interfaces. Somewhat annoyingly, the last operation gmsh does before a write tends to be a renumbering to sweep over 0D, 1D, 2D, and 3D objects sequentially. There has been some work on renumbering options recently, but it's targeted renumbering to make the indices dense rather than to allow other options.
On the other hand, the GMSH smh formats are very well documented and the ASCII text based ones are very easy to parse and rewrite. For what you want to do, the algorithm would go:
- parse the file once, storing an array of node values, and the element-node mappings.
- find the angles of each node from the centre of your polygon.
- Generate an integer array of the ordered node indices
- output a new file, outputting the node list in the new order and applying your index mapping to the element list.
This isn't totally trivial, but if you know a scripting language, wouldn't take forever to write.