There may be some mathematics or something obvious here but I don't really know where to start or what way to phrase my title/search.
Say there is a relationship Y = f(x) where f is a complicated non linear function. I have a model (g) that produces a Y* = g(x). Up until now I've chosen the parameters of g such that I minimize the squared error between Y and Y*. The problem I am running into is that I get solutions that may be good in some average sense "ie" they go through Y reasonably well, but along different subsections of x say if I split x into two [a,b) [b, c] the slope df(x)/dx vs dg(x)/dx may be wildly different say df(x)/dx is negative and dg(x)/dx is positive or flat. The trouble is that there are parameterizations of g that have more appropriate slopes in these regions that I would prefer.
One idea that I've thought of is defining some regions I care about (like the above) and doing a local correlation (to get the slope) for both the g and the f and adding an extra term to the optimization that penalizes say the square of this distance between these slopes. I have done things like this in the past when I know Y follows a fairly specific functional form with some error. With (a fair amount) of fine tuning I've found reasonable optimization schemes that work for the given problem. The current problem I'm working on has more complicated derivatives and although I could partition the space and penalize in a similar way, I'm wondering if this is the best approach.
Lastly there could be some non-parameteric style algorithms that could match Y well locally like LOESS, but g is constrained in a self consistent fashion that I need.
I've done some searching on non-linear optimization but I'm not really sure if that is the right path or that I can't find any resources I really understand. My question is generally if anyone has dealt with problems like this in the past, what did you do? Secondly are there good resources websites/papers/books to read about solving problems like this.
