Suppose I have a scan from an STM image (very much like the things you see here). Suppose I have a simple square lattice with lattice parameter a.
What I'd like to do is to numerically find the lattice parameter, measured in units of pixels, assuming that calibration is done elsewhere.
A first idea of mine was to have a function that creates a grid of points with lattice constant a, some offsets where the grid starts and also some angle for the rotation of the lattice. I'd then sum the values of the STM image at each grid point and return that. I'd then use some optimization toolbox (MATLAB, Python, ...) to find the parameters that maximize this sum.
Unfortunately, I run into problems doing this. For example, the grid points are calculated from the angle, lattice constant and offset and are then rounded to an integer value so that I can actually address my 2D image in the form
but most optimization routines will then go on to vary the parameters only very slightly, so that the rounded coordinates don't change. The program then assumes that it has found a local minimum/maximum since the function value doesn't change if the parameters are changed only slightly.
There are other problems regarding the stability of this method, so I wondered if there is a more sophisticated way of doing this in an automated fashion. I could always take the FT of the image data and read off the lattice constant manually, but I'd then still like to put a best-fid grid overlay over my image data so I'd have to optimize for angle and offset.