1
$\begingroup$

For fitting a response surface model to a physical process, I have 3-4 relevant "signals", like a feature density, a signal based on a feature width, or a signal based on a distance to the next feature.

The purpose of the surface model is to capture (=fit) the effect of inter-dependencies between the different signals on the response. I started with a bilinear model, where I have a constant term, a linear term for each signal, and a bilinear term for each product of two signals. This gave a nice fit overall, but I wondered how to add more fitting parameters for better fine tuning and non-physcial overfitting:

  1. I could add terms of higher polynomial degree
  2. I could use piecewise defined functions, something like piecewise multilinear interpolation or multilinear spline interpolation
  3. I could use rational functions instead of polynomial functions

I tried to fit the quotient of two bilinear model, and the result looked great. Maybe it's just because the many parameters allowed a non-physical overfitting, but now I'm burning to understand more about rational functions and their approximation properties anyway.

There are many possible parameterizations for rational functions, in addition to the simple quotient of two polynomial functions which I have used. The barycentric form is one possible parametrization of a univariate rational function. When I tried to better understand it, it turned out that it is closely related to the simple quotient of two polynomial functions. But other forms like allowing $\frac{1}{x}$ or $\frac{1}{1-x}$ terms (and some of their products with other terms) seem to be independent and closer related to higher polynomial functions.

Overall, I'm thinking about invariance of the form under transformations like $f(x,y)\to \alpha f(x,y)+\beta$ or $f(x,y)\to f(\alpha x+\beta, y)$. Now the simple quotient form is invariant under Möbious transformations like $f(x,y)\to \frac{a f(x,y)+b}{cf(x,y)+d}$, but that doesn't really seem like an important invariance.

More practically relevant seems the fact that any representation for polynomial functions directly translates into a corresponding representation for rational functions as a simple quotient. The number of parameters is at most doubled by this, which also seems like a good thing.

$\endgroup$
  • $\begingroup$ This is an interesting topic, but I'm not sure what the question is. When adding parameters, I always like to remember von Neumann's comment: "With four parameters I can fit an elephant, and with five I can make him wiggle his trunk as well." $\endgroup$ – Dave Kielpinski Feb 25 '15 at 8:07
  • $\begingroup$ @DaveKielpinski There are different parametrizations for polynomial functions, like monomial basis, Lagrange basis, Bernstein basis, all understood extremely well. Similar situation for piecewise polynomial functions. For rational functions, I have less experience and knowledge about the different parametrizations for rational functions. I indicated in the questions that I started to pick up that knowlege for univariate functions, but for multivariate functions, I would like to learn more... $\endgroup$ – Thomas Klimpel Feb 25 '15 at 10:59
2
$\begingroup$

What are good parametrizations of rational functions for response surface models?

A widely used flexible parametrization of (piecewise) rational functions are non-uniform rational basis spline (NURBS) models. The advantage of this parametrization is that the effect of the parameters can be understood intuitively. The effect of the denominator here is to assign a weight to each control point. (It allows to represent surfaces of revolution exactly without special treatment, which might be the main reason why basis spline models were generalized to NURBS models.)

Every piecewise univariate rational function can be represented by a NURBS model. Every (non-piecewise) multivariate rational function can be represented by a tensor product NURBS model. However, while the monomial basis representation of the quotient of two linear functions has $2n+1$ free parameters, the smallest tensor product NURBS model representation has $2\cdot 2^n$ parameters.

This is the difference between an $n$-simplex and an $n$-cube. If we give up the piecewise part, then we can define an analogous parametrization on an $n$-simplex by observing that the Bernstein polynomials arise from $1=(t+(1-t))^n=\sum_{j=0}^n\frac{n!}{j!(n-j)!}t^j(1-t)^{n-j}$, which is easy to generalize, for example the basis polynomials for a triangle arise from $1=(s+t+(1-s-t))^n=\sum_{i+j+k=n}\frac{n!}{i!j!k!}s^it^j(1-t)^k$.

The barycentric form is one possible parametrization of a univariate rational function.

It is know that any rational function $r(x)=\frac{p(x)}{q(x)}$ with $\max(\operatorname{deg}p,\operatorname{deg}q)\leq n$ for $n+1$ distinct interpolation points $x_j$ with $q(x_j)\neq 0$ can be written in barycentric form $$r(x)=\frac{\sum_{j=0}^n\frac{u_j}{x-x_j}r_j}{\sum_{j=0}^n\frac{u_j}{x-x_j}}$$ for $r_j:=r(x_j)$ and suitable weights $u_j$.

The tensor product barycentric form writes a rational function $r(x,y)=\frac{p(x,y)}{q(x,y)}$ with $\max(\operatorname{deg}_x p,\operatorname{deg}_x q)\leq m$ and $\max(\operatorname{deg}_y p,\operatorname{deg}_y q)\leq n$ as $$r(x,y)=\frac{\sum_{i=0}^m\sum_{j=0}^n\frac{u_{ij}}{(x-x_i)(y-y_j)}r_{ij}}{\sum_{i=0}^m\sum_{j=0}^n\frac{u_{ij}}{(x-x_i)(y-y_j)}}$$ for $m+1$ distinct $x$-positions $x_i$ and $n+1$ distinct $y$-positions $y_j$ with $q(x_i,y_j)\neq 0$ for $r_{ij}=r(x_i,y_j)$ and suitable weights $u_{ij}$. The proof remains essential identical to the univariate case: The barycentric Lagrange formula represents the denominator as $q(x,y)=l^m(x)l^n(y)\sum_{i=0}^m\sum_{j=0}^n\frac{w^m_iw^n_j}{(x-x_i)(y-y_j)}q_{ij}$ and $q_{ij}\neq0$ ensures $p(x,y)=\tilde{p}(x,y)$ for $\tilde{p}(x,y)=l^m(x)l^n(y)\sum_{i=0}^m\sum_{j=0}^n\frac{w^m_iw^n_j}{(x-x_i)(y-y_j)}q_{ij}r_{ij}$. Hence the weights $u_{ij}=w^m_iw^n_jq_{ij}$ lead to the barycentric representation.

This is an interesting topic, but I'm not sure what the question is. When adding parameters, I always like to remember von Neumann's comment: "With four parameters I can fit an elephant, and with five I can make him wiggle his trunk as well."

One implicit question I have is why rational functions are not considered a worthy topic for scicomp.SE or math.SE? You may object that I self-answered my question on math.SE. However, the answer is from a reference of a reference (I guess Nick Trefethen intentionally made the passing remark about rational functions, because they have been neglected indeed), and didn't answer the "What about the rational function $r(x,y)=\frac{xy}{x^2+y^2}$?" part, which is only answered above. This scicomp.SE question was more open ended, and also described the context, why I'm interested in parametrizations of rational functions. A reference to NURBS (or projective geometry) in the comments would have been helpful to put me on a track where additional "literature" and "hints" are available. I'm convinced that this answer is still much too focused on the few things about rational functions I could find myself, and misses some important bigger picture.

Overall, I'm thinking about invariance of the form under transformations like ...

Here are various transformations for which invariance of the form can be desirable:

  • $r(x,y,z)=f(x)+g(y,z)$
  • $r(x,y,z)=f(x)g(y,z)$
  • $r(x,y,z)=\alpha f(x,y,z)$
  • $r(x,y,z)=f(x,y,z)+\beta$
  • $r(x,y,z)=f(\alpha x,y,z)$
  • $r(x,y,z)=f(x+\beta,y,z)$

These invariances can be achieved by multilinear functions. But which additional invariances can be achieved by rational functions?

  1. $r(x,y,z)=\frac{f(x,y,z)}{1+\gamma f(x,y,z)}$
  2. $r(x,y,z)=f(\frac{x}{1+\gamma x},y,z)$

The simple quotient form is invariant under Möbious transformations like (1.), but that doesn't really seem like an important invariance. Let's check (2.) for $f(x,y)=\frac{c+c_x x+c_y y+c_{xy}xy}{1+d_x x+d_y y+d_{xy}xy}$ $$r(x,y)=\frac{c(1+\gamma x)+c_x x+c_y (1+\gamma x)y+c_{xy}xy}{1+\gamma x+d_x x+d_y(1+\gamma x)y+d_{xy}xy}$$ $$=\frac{c+(c_x+c\gamma)x+c_y y+(c_{xy}+c_y\gamma)xy}{1+(d_x+\gamma) x+d_y y+(d_{xy}+d_y\gamma)xy}$$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.