Some of the classic examples of ill-posed problems are to infer the coefficients of a PDE from measurements of the solution. For a specific example, consider the Poisson problem

$$-\nabla^2 u = f$$

on a domain $\Omega$ subject to the Dirichlet boundary condition $u|_{\partial\Omega} = 0$. The forward problem is to find a solution $u$ of this PDE that lives in the function space $L^2(\Omega)$ given the density $f$ which also lives in this space. The forward problem is well-posed, in that the mapping $f \to Gf$ is a continuous linear operator where $G$ is integration against the Green's function. For nice domains we can even compute explicit bounds on the operator norm of $G$.

Now suppose instead that we have a finite set of linear functionals $\{\mu_1, \ldots, \mu_N\}$ and we get some measurements

$$\xi_i = \langle \mu_i, u\rangle + \epsilon_i$$

where the $\epsilon_i$ are uncorrelated normal random variables with mean 0 and variance $\sigma_i$. Remember that we don't know what $u$ or $\epsilon$ is, just this vector $\xi$. Our job is to estimate the true value of $f$. We can turn this into a least-squares problem: find the minimizer $f$ of the quadratic functional

$$J(f) = \frac{1}{2}\|\xi - M\cdot G\cdot f\|_{\Sigma^{-1}}^2$$

where $M : L^2(\Omega) \to \mathbb{R}^N$ is the mapping from the observable field $u$ to the observations $\xi$. This inverse problem is ill-posed in that there are many possible solutions and a minute perturbation to the data $\xi$ results in a completely different estimated value of $f$.

To understand why the forward problem is well-posed but the inverse problem is not, it helps to understand the spectral characteristics of the Laplace operator. I said earlier that solving the Poisson equation is a continuous linear operation, but really I glossed over quite a bit of detail there. The solution operator $G$ is not just continuous -- it's a compact operator, and the output is much smoother than the inputs. By contrast, you can think of the Laplace operator itself as a high-pass filter, in that it tends to amplify high-wavenumber components of the input signal. Solving the inverse problem as stated amounts to applying this high-pass filter to noisy experimental data, which only amplifies the noise even more. This ill-posedness usually manifests itself in the form of extremely oscillatory or noisy estimates $f$ when attempting to solve the problem without some form of regularization. To speak specifically to the definition in the wikipedia article, the inverse operator is not continuous as a map from $L^2$ to itself. (It is continuous as a map from the Sobolev space $H^1$ to its dual $H^{-1}$ but that doesn't really help us any in practical terms.)

One way out of this dilemma is to use a Bayesian approach; usually we're not completely ignorant about $f$ and any prior information we might have can be used to regularize the problem. If you're interested in learning more about this, I really like the book Parameters Estimation and Inverse Problems -- Brian Borchers, who commented on Dan Doe's answer, is the second author.

Some of the classic examples of ill-posed problems are to infer the coefficients of a PDE from measurements of the solution. For a specific example, consider the Poisson problem

$$-\nabla^2 u = f$$

on a domain $\Omega$ subject to the Dirichlet boundary condition $u|_{\partial\Omega} = 0$. The forward problem is to find a solution $u$ of this PDE that lives in the function space $L^2(\Omega)$ given the density $f$ which also lives in this space. The forward problem is well-posed, in that the mapping $f \to Gf$ is a continuous linear operator where $G$ is integration against the Green's function. For nice domains we can even compute explicit bounds on the operator norm of $G$.

Now suppose instead that we have a finite set of linear functionals $\{\mu_1, \ldots, \mu_N\}$ and we get some measurements

$$\xi_i = \langle \mu_i, u\rangle + \epsilon_i$$

where the $\epsilon_i$ are uncorrelated normal random variables with mean 0 and variance $\sigma_i$. Remember that we don't know what $u$ or $\epsilon$ is, just this vector $\xi$. Our job is to estimate the true value of $f$. We can turn this into a least-squares problem: find the minimizer $f$ of the quadratic functional

$$J(f) = \frac{1}{2}\|\xi - M\cdot G\cdot f\|_{\Sigma^{-1}}^2$$

where $M : L^2(\Omega) \to \mathbb{R}^N$ is the mapping from the observable field $u$ to the observations $\xi$. This inverse problem is ill-posed in that there are many possible solutions and a minute perturbation to the data $\xi$ results in a completely different estimated value of $f$.

To understand why the forward problem is well-posed but the inverse problem is not, it helps to understand the spectral characteristics of the Laplace operator. I said earlier that solving the Poisson equation is a continuous linear operation, but really I glossed over quite a bit of detail there. The solution operator $G$ is not just continuous -- it's a compact operator, and the output is much smoother than the inputs. By contrast, you can think of the Laplace operator itself as a high-pass filter, in that it tends to amplify high-wavenumber components of the input signal. Solving the inverse problem as stated amounts to applying this high-pass filter to noisy experimental data, which only amplifies the noise even more. This ill-posedness usually manifests itself in the form of extremely oscillatory or noisy estimates $f$ when attempting to solve the problem without some form of regularization. To speak specifically to the definition in the wikipedia article, the inverse operator is not continuous as a map from $L^2$ to itself. (It is continuous as a map from the Sobolev space $H^1$ to its dual $H^{-1}$ but that doesn't really help us any in practical terms.)

One way out of this dilemma is to use a Bayesian approach; usually we're not completely ignorant about $f$ and any prior information we might have can be used to regularize the problem. If you're interested in learning more about this, I really like the book Parameter Estimation and Inverse Problems -- Brian Borchers, who commented on Dan Doe's answer, is the second author.

Some of the classic examples of ill-posed problems are to infer the coefficients of a PDE from measurements of the solution. For a specific example, consider the Poisson problem

$$-\nabla^2 u = f$$

on a domain $\Omega$ subject to the Dirichlet boundary condition $u|_{\partial\Omega} = 0$. The forward problem is to find a solution $u$ of this PDE that lives in the function space $L^2(\Omega)$ given the density $f$ which also lives in this space. The forward problem is well-posed, in that the mapping $f \to Gf$ is a continuous linear operator where $G$ is integration against the Green's function. For nice domains we can even compute explicit bounds on the operator norm of $G$.

Now suppose instead that we have a finite set of linear functionals $\{\mu_1, \ldots, \mu_N\}$ and we get some measurements

$$\xi_i = \langle \mu_i, u\rangle + \epsilon_i$$

where the $\epsilon_i$ are uncorrelated normal random variables with mean 0 and variance $\sigma_i$. Remember that we don't know what $u$ or $\epsilon$ is, just this vector $\xi$. Our job is to estimate the true value of $f$. We can turn this into a least-squares problem: find the minimizer $f$ of the quadratic functional

$$J(f) = \frac{1}{2}\|\xi - M\cdot G\cdot f\|_{\Sigma^{-1}}^2$$

where $M : L^2(\Omega) \to \mathbb{R}^N$ is the mapping from the observable field $u$ to the observations $\xi$. This inverse problem is ill-posed in that there are many possible solutions and a minute perturbation to the data $\xi$ results in a completely different estimated value of $f$.

To understand why the forward problem is ill-posed but the inverse problem is not, it helps to understand the spectral characteristics of the Laplace operator. I said earlier that solving the Poisson equation is a continuous linear operation, but really I glossed over quite a bit of detail there. The solution operator $G$ is not just continuous -- it's a compact operator, and the output is much smoother than the inputs. By contrast, you can think of the Laplace operator itself as a high-pass filter, in that it tends to amplify high-wavenumber components of the input signal. Solving the inverse problem as stated amounts to applying this high-pass filter to noisy experimental data, which only amplifies the noise even more. This ill-posedness usually manifests itself in the form of extremely oscillatory or noisy estimates $f$ when attempting to solve the problem without some form of regularization. To speak specifically to the definition in the wikipedia article, the inverse operator is not continuous as a map from $L^2$ to itself. (It is continuous as a map from the Sobolev space $H^1$ to its dual $H^{-1}$ but that doesn't really help us any in practical terms.)

One way out of this dilemma is to use a Bayesian approach; usually we're not completely ignorant about $f$ and any prior information we might have can be used to regularize the problem. If you're interested in learning more about this, I really like the book Parameters Estimation and Inverse Problems -- Brian Borchers, who commented on Dan Doe's answer, is the second author.

Some of the classic examples of ill-posed problems are to infer the coefficients of a PDE from measurements of the solution. For a specific example, consider the Poisson problem

$$-\nabla^2 u = f$$

on a domain $\Omega$ subject to the Dirichlet boundary condition $u|_{\partial\Omega} = 0$. The forward problem is to find a solution $u$ of this PDE that lives in the function space $L^2(\Omega)$ given the density $f$ which also lives in this space. The forward problem is well-posed, in that the mapping $f \to Gf$ is a continuous linear operator where $G$ is integration against the Green's function. For nice domains we can even compute explicit bounds on the operator norm of $G$.

Now suppose instead that we have a finite set of linear functionals $\{\mu_1, \ldots, \mu_N\}$ and we get some measurements

$$\xi_i = \langle \mu_i, u\rangle + \epsilon_i$$

where the $\epsilon_i$ are uncorrelated normal random variables with mean 0 and variance $\sigma_i$. Remember that we don't know what $u$ or $\epsilon$ is, just this vector $\xi$. Our job is to estimate the true value of $f$. We can turn this into a least-squares problem: find the minimizer $f$ of the quadratic functional

$$J(f) = \frac{1}{2}\|\xi - M\cdot G\cdot f\|_{\Sigma^{-1}}^2$$

where $M : L^2(\Omega) \to \mathbb{R}^N$ is the mapping from the observable field $u$ to the observations $\xi$. This inverse problem is ill-posed in that there are many possible solutions and a minute perturbation to the data $\xi$ results in a completely different estimated value of $f$.

To understand why the forward problem is well-posed but the inverse problem is not, it helps to understand the spectral characteristics of the Laplace operator. I said earlier that solving the Poisson equation is a continuous linear operation, but really I glossed over quite a bit of detail there. The solution operator $G$ is not just continuous -- it's a compact operator, and the output is much smoother than the inputs. By contrast, you can think of the Laplace operator itself as a high-pass filter, in that it tends to amplify high-wavenumber components of the input signal. Solving the inverse problem as stated amounts to applying this high-pass filter to noisy experimental data, which only amplifies the noise even more. This ill-posedness usually manifests itself in the form of extremely oscillatory or noisy estimates $f$ when attempting to solve the problem without some form of regularization. To speak specifically to the definition in the wikipedia article, the inverse operator is not continuous as a map from $L^2$ to itself. (It is continuous as a map from the Sobolev space $H^1$ to its dual $H^{-1}$ but that doesn't really help us any in practical terms.)

One way out of this dilemma is to use a Bayesian approach; usually we're not completely ignorant about $f$ and any prior information we might have can be used to regularize the problem. If you're interested in learning more about this, I really like the book Parameters Estimation and Inverse Problems -- Brian Borchers, who commented on Dan Doe's answer, is the second author.