12

Check out this paper by Des Higham and the SDETools MATLAB toolbox.


10

This is the stochastic root-finding problem, as in The stochastic root-finding problem: Overview, solutions, and open questions.


8

There is a theorem that syas that a black box algorithm is guaranteed to find the global minimum of an arbitrary smooth (i.e., twice continuously differentiable) function if and only if it samples points densely in the search space. Here dense is meant in the topological sense, i.e., it must sample points in arbirarily small neighborhoods of every point. ...


8

You can generate a noise sequence with whatever noise spectrum you want (including $1/f$, also known as pink noise) by generating the noise coefficients in spectral space. The magnitudes of the coefficients should be chosen to give the desired spectrum and the phases should be chosen randomly. You then simply perform an inverse Fourier transform to give the ...


7

I can understand that this property is useful in some applications where the derivative is difficult or computationally infeasible to obtain or does not exist. However, I would not expect such problems to be very relevant in application. If an analytical solution for the derivative is not known, it's very costly and error prone. Calculating the Jacobian is $...


5

Since $\phi$ is a scalar between $0$ and $1$, the easiest method for finding a root is bisection. If you cannot calculate the expectation of the nonlinear function $$f_\phi(\theta) = \left(\phi r_z +(1-\phi)(r_k+\theta)\right)^{1-\gamma}(r_k+\theta-r_z)$$ in terms of $\phi$ analytically, you can use quadrature to approximate. For the first variant, this ...


5

I have just corresponded with the author of the software, and my suspicions were confirmed. The spatial stepsize for $n$ equispaced intervals for this problem should be $$\frac{x_{n+1}-x_0 }{n}=\frac{y_{n+1}-y_0}{n}=\frac{2}{n}$$.


5

The Wikipedia article on this topic is rather bad (from an educational point of view) and somewhat misleading. Do not use it. You'll find a better summary of the method here. In particular, I find it confusing to call $a_k$ a step sequence. To get an intuitive understanding of the method, let's look at Newton's method for the (deterministic) function $M(...


5

In the deterministic case, you can of course run the same algorithm, but it is very inefficient compared to quasi-Newton (Broyden type) methods. There is little point to investigate the properties of so poor an algorithm. On the other hand, broyden's method is quite sensitive to noise, hence cannot be easily adapted to the stochastic case. Moreover, if the ...


5

The term "mean square" is usually used when one wants to minimize a quantity that can be either positive or negative. Consider a series of values $x_i$ for $i = 1, \ldots, N$. If the $x_i$ are all large positive or large negative numbers, then the average value $\left< x \right>$ of the $x_i$ could still be nearly zero, even though none of the ...


5

Response surface models (a kind of surrogate model) are often used in situations like this. The idea is to sample values of the parameters $a$ and $b$, compute $R(a,b)$ at each point, and then build a regression model (typically quadratic or even higher order) of the function. You can then optimize over the the fitted model.


4

No, it is not. Very few methods are provably convergent for nonconvex optimization problems. If you're looking for such a method, you might look into branch-and-bound methods.


4

Programs like Visit and Paraview can do "volume rendering", which is what you show in your figure. You just need to export the data you have in a format that either of these programs can read.


4

I am not an expert in specifically stochastic differential equations, but I would assume that my answer will still be of some value. Computation of the derivative can be challenging, as you mentioned in your question. However, this would be even more pronounced in a multidimensional case, as one would have to calculate Jacobian matrices ($n^2$ entries). So, ...


3

There is a nice volume rendering toolbox for MATLAB: http://www.mathworks.com/matlabcentral/fileexchange/22940-vol3d-v2 I think you could tweak it for your purposes.


3

Depending on your specific system and the size, you could try a line search method as suggested in the other answer such as Conjugate Gradients to determine step size. However, if your data size is really large, this might become very inefficient and time consuming. For large datasets people often choose a fixed step size and stop after a certain number of ...


3

The most straightforward way to solve your SDE is with an Euler-Maruyama scheme. This is a simple and effective method for additive noise, i.e., the diffusion/noise term is not a function of the state, as appears to be the case for your example. Here is some Matlab code to solve your system: n = 2; % Order of system t0 = 0; % Initial time dt = ...


3

There is a huge literature on "stochastic programming", but you're probably interested in what is called "chance constrained programming", in which the constraint coefficients are random variables, and you want to find a solution $x$ such that each constraint is individually satisfied with probability $\eta$ ($\eta$ might 0.95, 0.99, etc.) These problems ...


3

You're interested in the solution $u(x,t)$ of the heat equation subject to some boundary conditions and an initial condition $u(x,0)=\delta(x-x_{0})$. You can approximate this at times $t_{1}$, $t_{2}$, $\ldots$, $t_{n}$ by simulating the diffusion of a bunch of particles from $x_{0}$ and constructing histograms of the particle distributions at times $t_{1}$...


3

There is no reason to believe that two random fields with the same arithmetic mean would yield solutions that have anything to do with each other. In fact, for the case you consider, one might imagine that maybe the harmonic mean is actually a better indicator, but even that is unclear -- it could also be the geometric mean. Apart from this, you have to ...


2

If you adopt the sparse grid approach, you will have to choose between "intrusive" stochastic Galerkin and "unintrusive" stochastic collocation. The former performs "mixing" of your approximation of uncertainty using the operators in the model, thus potentially converging faster than the latter which performs independent model evaluations. The cost of this ...


2

Another suggestion via a blog post I just read that refers to a talk at NIPS 2007 tutorial: Do a line search once for a subset of samples, and then fix the learning rate to be the effective step size within the line search.


2

You're probably better off switching to a deterministic method instead of simulated annealing. Provided you can import the data to a text file, you can learn the basics of the algebraic modeling language GAMS without too much effort, and use the BARON solver to attempt to solve your problem. BARON is a very good (though not foolproof) nonconvex nonlinear ...


2

If $H = H(q, p)$ is the Hamiltonian of your mechanical system, $T$ is the temperature and $\gamma$ is the friction, then Langevin dynamics (LD), \begin{equation*} \left\{ \begin{aligned} \mathrm{d} q &= \nabla_p H(q, p) \, \mathrm{d} t, \\ \mathrm{d} p &= -\nabla_q H(q, p) \, \mathrm{d} t - \gamma \, p \, \mathrm{d} t + \sqrt{2 \...


2

You are applying Richardson extrapolation to increasingly-accurate independent sample paths of an OU process. How would this work even for independent identically normally distributed random variables $y_1,y_2,y_3,y_4\sim\mathrm{N}(\mu,\sigma^2)$? These have already "converged" to the true distribution $\mathrm{N}(\mu,\sigma^2)$, and you might expect the ...


2

Active learning (aka experimental design) strategies are suitable for this. One of them being the response surface modeling suggested by Brian Borchers. The idea is to choose the next point to evaluate to learn about the optimal M as fast as possible. Here are some old and new papers to start from: Harold J. Kushner. A new method of locating the maximum of ...


2

(1.8) is a simple reformulation of the deterministic LP (1.2) It's still a deterministic LP. This may be somewhat confusing since the authors are going back forth between a deterministic LP formulation (assuming that $d$ is known) and a stochastic LP formulation (in which we're minimizing the expected value over random values of $d$.) See (1.9) for the ...


2

How is the KDE used for calculating the new residual value here? Assume that you have already processed $n$ samples $x_1,x_2,\ldots,x_n \in \mathbb{R}^3$. Then the residual images $r_j : \mathbb{R}^2 \rightarrow \mathbb{R}$, $j=1,2,\ldots$, take the form $$r_j(x) = m_j(x) - \sum_{i=1}^n e_i K(x-proj_j(x_i))$$ where $m_j$ is the $j$-th projection image, ...


2

First of all, it is impossible to intertwine a multi-step Runge–Kutta method and the Milstein–Itō methods for a multitude of reasons that go beyond the scope of this question¹. So the best you can possibly do in is: Make a deterministic Runge–Kutta step, ignoring the noise term. Apply noise (in a Milstein–Itō fashion). Go to 1. This has the problem that ...


2

You cannot take deterministic methods and slap a kind of Milstein correction on them to get higher order. Instead, you should be looking at methods which are specifically designed for integrating SDEs at high order if you want good accuracy. If pure efficiency is what matters and the noise is additive/diagonal, then the adaptive Rossler methods mentioned by @...


Only top voted, non community-wiki answers of a minimum length are eligible