1
$\begingroup$

This is related to my previous post here

I have a dataset with values of multiple curves. An example plot is shown below. I want to scale the curves (move up/down) so that all curves overlap.

The following is a sample dataset which includes that data points corresponding to 5 curves and coordinate inputs below

scale =  1.5;
x1 = [0,4,6,10,15,20]*scale;
y1 =  [18,17.5,13,12,8,10];
x2 = [0,10.5,28]*scale;
y2= [18.2,10.6,10.3];
x3 = [0,4,6,10,15,20]*scale;
y3 = [18,13,15,12,11,9.6];
x4 = [9,17,28]*scale;
y4 = [5,5.5,7];
x5 = [1,10,20]*scale;
y5 = [3,0.8,2];

plot(x1,y1, '*-', x2, y2,  '*-', x3, y3,  '*-', x4, y4,  '*-', x5, y5,  '*-')

enter image description here

To scale the curves, I need to find the scale factor by defining a target curve. I'm not sure of the ways in which the target curve can be defined. Would it be a good approach to compute the weighted average? Since the x scale is different for each curve, I am not sure how to define an average/ target curve.

Suggestions will be really appreciated.

$\endgroup$
8
  • $\begingroup$ I'm not sure how you define the "sum of a curve". Do you mean the area under the curve? Or some norm like $\|f\|_2=\int f(x)^2\;dx$? $\endgroup$
    – PC1
    Jul 5, 2022 at 20:33
  • 1
    $\begingroup$ Subtract the mean of each curve. Optionally scale each so its RMS = 1. $\endgroup$ Jul 5, 2022 at 21:43
  • $\begingroup$ @PC1 Thanks for the reply. I mean the average of curves. Please let me know if I am still unclear. $\endgroup$
    – Natasha
    Jul 6, 2022 at 7:34
  • $\begingroup$ @Aruralreader Thanks for the reply. Should I compute the mean of each curve and subtract every data point in the curve from the mean? I'm not sure how to scale so that RMS=1. Could you please explain this a bit? $\endgroup$
    – Natasha
    Jul 6, 2022 at 9:42
  • $\begingroup$ I would first find a coordinate $x_0$ that all curves intervals include, and then interpolate each curve to that $x_0$. The interpolated value would be the scaling factor for each curve. $\endgroup$ Jul 6, 2022 at 14:24

2 Answers 2

2
+100
$\begingroup$

I promised you an answer in the other question, and was just about to edit it in. Now I see you spend another 100 points as a bounty ... seems quite a serious topic to you. I'll post my promised answer, and there's no need to spend the other 100 points on me ... for what it's worth.

Ok, let's start where we have been in the other thread, and summarize. You'll need:

  1. A measure of distance between curves. Otherwise, there is no meaning to set up a target function. One choice mentioned in the other thread was to do an interpolation of the data $\{f_i\}_{i=1}^M$ on a common, and probably finer, set of gridpoints $x_k$, which will produce functions $f_i(x)$, and then simply use the squared difference evaluated on the grid, $$ D(f_i,f_j) = \sum_{k=1}^N | f_i(x_k) - f_j(x_k)|^2 $$ As interpolation you can use piecewise linear, polynomial, spline, or any other reasonable choice.

  2. A model on how to scale the curves. Let's assume for the moment it is parametrized by a single parameter $\alpha_i$ and produces a function $$ \tilde f_i(x,\alpha_i) $$ In the other thread you seemd to prefer amultiplicative function -- more on that later in this post.

  3. A functional what to minimize. In the other thread, we chose a single curve as reference and adjusted the others with respect to thier parameters $\alpha_i$ as to come as close to the reference as possible. In this case, one could simply use the distance $D(\tilde f_i(\alpha_i),f_j)$ as target function. Now, you want all that all curves are scaled at the same time, and so you'll lose the reference curve. With this you could use $$ L(\alpha_i;\lambda) = \sum_{i<j}^M \sum_{k=1}^N | \tilde f_i(x_k,\alpha_i) - \tilde f_j(x_k,\alpha_j)|^2 + \text{some appropriate penalty term on } \alpha_i $$ I've added a penalty term here which imposes that the parameters should be as small as possible. It is necessary because otherwise, depending on the model for the functions, the problem is not uniquely defined (see below)
    The penalty term could be $\lambda \sum_{i=1}^M| \alpha_i|^2$, for example, with which you would obtain a method called ridge regression. Now you can pick a $\lambda$ optimize the $\{\alpha_i\}_{i=1}^M$ in order to minimize the functional $L(\alpha_i;\lambda)$.

I'll apply this to some simple models in the following. Everything from here is simple calculus, where I hope I didn't make too many mistakes.


Additive model:

Let's first apply the concepts above to the model $\tilde f_i(x) = f_i(x) + \alpha_i$. The penalty function is the one from ridge regression (if there would be no such penalty function, for any solution ${\alpha_i}_{i=1}^M$, ${\alpha_i + c}_{i=1}^M$ would also be a solution.

Upon insertion into the functional, one obtains $$ L(\alpha_i;\lambda) = \sum_{i<j}^M \sum_{k=1}^N | f_i(x_k) + \alpha_i - \tilde f_j(x_k) - \alpha_j|^2 + \lambda \sum_{i=1}^M| \alpha_i|^2 $$ and thus $$ \partial_{\alpha_i} L = 2\sum_{j=i+1}^M \sum_{k=1}^N \left(f_i(x_k) + \alpha_i - f_j(x_k) - \alpha_j\right)+ 2\lambda \alpha_i\\ = \left(\sum_{k=1}^N (f_i(x_k)-\sum_{j=i+1}^M f_j(x_k)\right) + (1 + \lambda) \alpha_i - \left(\sum_{j=i+1}^M \alpha_j \right) = 0 $$ or $$ \alpha_i= \frac{1}{1+\lambda} \left( \left(\sum_{j=i+1}^M \alpha_j \right) - \left(f_i(x_k)-\sum_{j=i+1}^M f_j(x_k)\right) \right) $$

You can solve this iteratively starting from $i=M$ and going down to $i=1$.


Multiplicative model:

Using the model $\tilde f_i(x) = \alpha_i f_i(x)$. I'll do that some other day, but just want to note, that you need another for of penalty function here, for example $\lambda_i \sum_i \log(\alpha_i)$. Otherwise, e.g. in the ridge regression case before, the global solution would be $\alpha_i=0$, which is likely not what you want.

$\endgroup$
3
  • $\begingroup$ Thanks a lot for the detailed explanation. I'll apply this to a dataset and update my observations here. It has always happened that the questions with bounties are explained in great detail and in a short span :) and I'm happy to spend my points on the solutions posted by experts in the field for the amount of time that is invested:) .Also, if I don't award the bounty I lose points anyway :D $\endgroup$
    – Natasha
    Jul 14, 2022 at 16:59
  • 1
    $\begingroup$ @Natasha: may I ask -- what is the ultimate goal of this task? In which field do you work, and what kind of data is this? What do you win when the curves overlap? $\endgroup$
    – davidhigh
    Jul 14, 2022 at 17:18
  • $\begingroup$ Sure, I am using datasets for predictive modeling in biology. The data comes from different sources and is often reported in arbitrary units. In such cases, I am trying to see if this approach can help in scaling the measurements. $\endgroup$
    – Natasha
    Jul 15, 2022 at 16:35
0
$\begingroup$

Would it be a good approach to compute the weighted average?

I don't think it would be a good choice (in biology, as marked in comments.) in any case. what is $x$ & what is $y$?

but what for would you like to put-up/down curves to the same level, if they will still remain to be different curves in your analysis?..

I can see from such data opportunities for these 2 sorts of analysis only:

  1. you can either find $\frac{dy}{dx}$ approximation of slope for each curve - to make conclusion about slope (effect) size (preferably for the same x-values in laboratory or not the same like in reality you have - but in such a case is not understood what you're investigating - if $y$ as effect/result of $x$ exposure?) -- slope, anyway, could show the effect-size (or sensitivity)

  2. or analyse $x-\overline{x}$ to conclude about variation for each sample

$\endgroup$
2
  • $\begingroup$ Welcome to scicomp. A tip: You can use MathJax to typeset your mathematical formulas. This will make the question much easier to read. $\endgroup$ Jul 10, 2023 at 10:22
  • $\begingroup$ Thanks, I corrected $\endgroup$
    – JeeyCi
    Jul 11, 2023 at 16:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.