# How to determine the orientation of convex/concave hexahedra?

I am writing a code that checks the orientation of a list of vertices (along with face connectivity) describing both convex and concave hexahedra. The face connectivity table stores the list of vertex indices that constitute each face, as described on p.27 of this document. I flag a hexahedron as being positively oriented if all faces are oriented (i.e. clockwise or counter-clockwise) such that the face normal points outward. The direction of the face normal can be determined by applying the right-hand rule in the plane of the face: four fingers curl in the direction of the face vertex orientation, and the thumb points in the direction of the face normal.

I also store the vertex list in the same order as in the linked document. However, the issue I have is that there are certain geometric operations (e.g. reflection about a plane) in my code that flip the orientation of a given hexahedron. I am currently able to check the orientation robustly for convex hexes by computing the centroid and ensuring that the face normals are outward-pointing with respect to it. However, this is not a robust procedure in the case of concave hexes because - (i) the centroid may lie outside the hex, or (ii) the interior side of a face may not be 'visible' to the centroid.

Can anyone suggest a robust procedure for checking the orientation of arbitrary hexahedra? Any advice would be greatly appreciated.

• Generally one is not interested in the absolute orientation of an element. The mesh connectivity can uniquely be defined by considering only changes of the orientation between two neighbouring elements. Commented Jul 30, 2021 at 11:11
• I agree, but in my context, the CFD code assumes that all elements are oriented as per Gambit Neutral format. The mesh is pre-processed under that assumption. However, we have found that some mesh generators don't necessarily conform to Gambit neutrality (e.g. after performing reflections in the mesh generation software). So a collection of elements with inconsistent orientations messes up pre-processing. Therefore, my code has to detect and enforce Gambit Neutrality of all elements. Convex elements are fine. My problem lies in detecting Gambit neutrality of concave elements. Commented Jul 30, 2021 at 11:30
• If you have a unique side list, then simply loop over all sides. Choose a first element/side, where the orientation is known. Check the first point of sides sharing two elements. This has to be done consecutively. Commented Jul 30, 2021 at 11:43
• Simply use the data structure. Do not try to solve the problem geometrically. Commented Jul 30, 2021 at 11:48