3
$\begingroup$

I was reading the Wikipedia page for "Well-posed problems". Supposedly, if a problem is well-posed, it must meet the following conditions:

  • a solution exists
  • the solution is unique
  • the solution's behavior changes continuously with the initial conditions

I can understand why the first two conditions are desirable for a problem to be well-posed, but I am having trouble understanding the last condition.

For instance, if "the solution's behavior changes continuously with the initial conditions" - isn't this a bad thing? Would we not want the "solution's behavior NOT TO CHANGE continuously with the initial conditions"?

If the solution's behavior can change with the initial conditions, would this not result in the problem having the ability to be "chaotic and unpredictable", thus displaying "bad behavior" and as a result — ill-posed?

$\endgroup$
2
  • $\begingroup$ What they are talking about is to make sure the behavior of the solution does not change discontinuously with the change of initial conditions. Changing continuously (which includes no change at all) is ok. If the solution does change with the initial conditions (and nontrivial systems are like that), there is division into deterministic and non-deterministic systems. For example a 2-body system interacting by gravity always has deterministic solutions. For a 3-body system, two solutions, slightly different initially, can be exponentially diverging in time, those are non-deterministic. $\endgroup$ Feb 6, 2022 at 2:45
  • 3
    $\begingroup$ "Continuous" here is referring to mathematical continuity. Colloquially it means small changes in the input data should only result in small changes in output result. $\endgroup$ Feb 6, 2022 at 4:10

3 Answers 3

7
$\begingroup$

Continuous dependence on initial conditions means that if for every $\epsilon > 0$ there exists a $\delta(\epsilon) > 0 $ such that $$\Vert x_0 - \widetilde{x_0} \Vert \leq \delta(\epsilon) \Rightarrow \Vert x(t, x_0) - x(t, \widetilde{x_0}) \Vert \leq \epsilon \: \forall \: t_0 = 0 \leq t \leq T$$ Where I used the definition given here with $t_0 = 0$.

The classic example why this matters is the Lorenz System/Attractor which shows at least sensitive dependence on initial conditions. A nice intro to the topic is this blog post where it is shown that the Lorenz ODEs show very different behaviour for seemingly small deviations in the initial condition.

Speaking of chaotic dynamical systems, you can also have chaotic behaviour due to changes in the parameters $\mu$ of the system. Thus, your definition of well-posedness for parametric systems $f\big(x(t), t ; \mu\big)$ might be extended to requiring also continuous dependence on the parameters.

Why are these continuous dependence criteria required for a well-posed problem? From a computational science standpoint, you want to trust the solutions you obtain from simulations. The issue is that every computer can represent initial conditions only up to finite accuracy - any differences below the threshold of the datatype can not be realized. However, since you can also measure actual physical quantities only up to certain accuracy, you will in practice never exactly hit the true physical outset. For systems with no continuous dependence on initial data, you can thus basically not trust your simulation! This is why for instance in weather simulations people run multiple simulations with different initial conditions and obtain an average to predict whats likely going to happen.

$\endgroup$
4
  • 1
    $\begingroup$ I'm not sure about the claim that the Lorenz system fails to depend continuously on initial conditions. It is certainly very sensitive, but because the RHS function is Lipschitz continuous, the problem should be well-posed. See, for example, folk.ntnu.no/hanche/kurs/dynsys/2007v/wellposed.pdf $\endgroup$ Feb 6, 2022 at 15:09
  • 1
    $\begingroup$ I do not think that the Lorenz system is globally Lipschitz due to the nonlinear terms. But you are right, sensitive dependence on inital conditions is the less bold claim. $\endgroup$
    – Dan Doe
    Feb 6, 2022 at 15:25
  • $\begingroup$ More precise, bilinear terms. $\endgroup$
    – Dan Doe
    Feb 6, 2022 at 15:49
  • $\begingroup$ In addition to the limitations of floating point arithmetic, it's usually the case that there are model inputs that come from noisy experimental data. If there is sensitive dependence on those inputs, then you can get useless outputs. $\endgroup$ Feb 6, 2022 at 19:01
3
$\begingroup$

Some of the classic examples of ill-posed problems are to infer the coefficients of a PDE from measurements of the solution. For a specific example, consider the Poisson problem

$$-\nabla^2 u = f$$

on a domain $\Omega$ subject to the Dirichlet boundary condition $u|_{\partial\Omega} = 0$. The forward problem is to find a solution $u$ of this PDE that lives in the function space $L^2(\Omega)$ given the density $f$ which also lives in this space. The forward problem is well-posed, in that the mapping $f \to Gf$ is a continuous linear operator where $G$ is integration against the Green's function. For nice domains we can even compute explicit bounds on the operator norm of $G$.

Now suppose instead that we have a finite set of linear functionals $\{\mu_1, \ldots, \mu_N\}$ and we get some measurements

$$\xi_i = \langle \mu_i, u\rangle + \epsilon_i$$

where the $\epsilon_i$ are uncorrelated normal random variables with mean 0 and variance $\sigma_i$. Remember that we don't know what $u$ or $\epsilon$ is, just this vector $\xi$. Our job is to estimate the true value of $f$. We can turn this into a least-squares problem: find the minimizer $f$ of the quadratic functional

$$J(f) = \frac{1}{2}\|\xi - M\cdot G\cdot f\|_{\Sigma^{-1}}^2$$

where $M : L^2(\Omega) \to \mathbb{R}^N$ is the mapping from the observable field $u$ to the observations $\xi$. This inverse problem is ill-posed in that there are many possible solutions and a minute perturbation to the data $\xi$ results in a completely different estimated value of $f$.

To understand why the forward problem is well-posed but the inverse problem is not, it helps to understand the spectral characteristics of the Laplace operator. I said earlier that solving the Poisson equation is a continuous linear operation, but really I glossed over quite a bit of detail there. The solution operator $G$ is not just continuous -- it's a compact operator, and the output is much smoother than the inputs. By contrast, you can think of the Laplace operator itself as a high-pass filter, in that it tends to amplify high-wavenumber components of the input signal. Solving the inverse problem as stated amounts to applying this high-pass filter to noisy experimental data, which only amplifies the noise even more. This ill-posedness usually manifests itself in the form of extremely oscillatory or noisy estimates $f$ when attempting to solve the problem without some form of regularization. To speak specifically to the definition in the wikipedia article, the inverse operator is not continuous as a map from $L^2$ to itself. (It is continuous as a map from the Sobolev space $H^1$ to its dual $H^{-1}$ but that doesn't really help us any in practical terms.)

One way out of this dilemma is to use a Bayesian approach; usually we're not completely ignorant about $f$ and any prior information we might have can be used to regularize the problem. If you're interested in learning more about this, I really like the book Parameter Estimation and Inverse Problems -- Brian Borchers, who commented on Dan Doe's answer, is the second author.

$\endgroup$
2
  • 1
    $\begingroup$ Looks like a typo here: "To understand why the forward problem is ill-posed but the inverse problem is not, it helps to understand the spectral characteristics of the Laplace operator." $\endgroup$ Feb 7, 2022 at 3:16
  • 1
    $\begingroup$ Yes thanks for the catch @MaximUmansky, just fixed it! $\endgroup$ Feb 7, 2022 at 16:49
1
$\begingroup$

I suspect you are confusing well-posed optimization problem (which you would want one solution to, regardless of initial guess) and well-posed differential equation (which you would want to be continuous in the initial conditions)

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.