20 votes
Accepted

Use of machine learning in computational fluid dynamics

It's a long-running joke that CFD stands for "colorful fluid dynamics". Nevertheless, it is used -- and useful -- in a wide range of applications. I believe your discontent stems from not ...
user avatar
20 votes
Accepted

Sensitivity of $y$ w.r.t. to $x$ in $y=f(x)$ where f is a routine

The act of using automatic differentiation for derivative calculations of general programs has become known as differentiable programming. There are many grades of differentiable programming across ...
user avatar
17 votes
Accepted

Why aren't Krylov subspace methods popular in the Machine Learning community compared to Gradient Descent?

On a basic level, I don't buy the argument that you have to "solve a linear system for many machine learning algorithms". Much more, you usually have to optimize a non-linear equation which ...
user avatar
  • 2,547
11 votes
Accepted

Markov (Chain) image generators?

I've implemented this recently, basically it counts how many times each specific colour borders another colour to make up a frequency table. To generate an image, a random colour and position are ...
user avatar
  • 226
11 votes

Are there tasks in machine learning which require double precision floating points?

I am not an expert in machine learning, but I can outline the considerations that are relevant. The numerical calculations in machine learning are generally linear algebra -- either solving linear ...
user avatar
7 votes
Accepted

Learning parameters of noise and filter coefficients from data where data and noise both have Gaussian distributions

So the way I went about formulating the problem was to essentially write the following equations: The state that will be estimated, which is defined as a column vector, is the following: $$w = [vec(A)...
user avatar
  • 3,673
7 votes

Use of machine learning in computational fluid dynamics

I think you are mixing a couple different ideas that are causing confusion. Yes, there are a wide variety of ways to discretize a given problem. Choosing an appropriate way may look like "voodoo" ...
user avatar
  • 4,567
7 votes

Seeking a free symbolic regression software

I wrote a Python package called PyPGE. PyPGE is a Symbolic Regression implementation based on Prioritized Grammar Enumeration (1), not Evolutionary or Genetic Programming. It produces a deterministic ...
user avatar
  • 179
7 votes
Accepted

What are the benefits of using machine learning for interpolation over traditional interpolation methods?

First of all, interpolation and approximation are slightly different from each other. Given a sufficiently smooth function $f$ (sufficiently smooth just means that I am covering my bases, there are ...
user avatar
6 votes
Accepted

When training a neural network, why choose Adam over L-BGFS for the optimizer?

This topic has been discussed at some length on Cross Validated (aka stats.stackexchange) and Reddit: Why is Newton's method not widely used in machine learning? (see in particular Nick Alger's ...
user avatar
  • 2,106
6 votes
Accepted

Artificial Intelligence, Modeling and Simulation

So bottom line is I don't see any comprehensive work on the use of AI in M&S as a whole, let's say having models that can learn how to produce new improved models using the existing models. There'...
user avatar
5 votes

Why am I not seeing faster neural network training after upgrading to a vastly better GPU?

I suppose, you right and your network is not that big to 100%-utilize the GPU. The bottle-neck here seems to be not the GPU itself, but the transfer rate between RAM and VRAM and here the difference ...
user avatar
  • 114
5 votes
Accepted

Machine Learning for Optimization

The general idea that you have of learning an easy to compute model from results of your detailed simulation model and then optimizing the easy to compute model is long-established. The easy to ...
user avatar
5 votes
Accepted

Advantage of diagonal "jitter" for numerical stability?

So, you want to invert your matrix $A=\Phi^T\Phi$. For $A$ to be invertible it must not have zero eigenvalues. We can show that $A$ is positive semi-definite as follows. Positive semi-definite means ...
user avatar
  • 428
5 votes

Why aren't Krylov subspace methods popular in the Machine Learning community compared to Gradient Descent?

They aren't popular because they don't work. Nicol N. Schraudolph spent a few years on Krylov-like methods for Machine Learning. I first learned of his work at 2004 Machine Learning Summer School in ...
user avatar
4 votes

Seeking a free symbolic regression software

I once started writing anopen source version of Eureqa in Java. The project has limited capabilities but it implements the fitness function described in [1] and couple optimizations mentioned by the ...
user avatar
4 votes

Seeking a free symbolic regression software

After a cursory google search on the subject, it appears that "symbolic regression" is a problem that lends itself greatly to stochastic optimization algorithms like genetic programming (GP). It is ...
user avatar
  • 11.8k
4 votes

Advantage of diagonal "jitter" for numerical stability?

Think of the simplest case when $\Phi$ is a scalar value. Not well defined: $$ \boldsymbol \theta^\text{ML} = (0^T 0)^{-1}0^T ~ y = \frac{1}{0} 0~y= \frac{0}{0} $$ Well defined: $$ \boldsymbol \theta^...
user avatar
4 votes

Advantage of diagonal "jitter" for numerical stability?

Look up something on Tikhonov regularization, also known as ridge regression in machine learning. This is a standard technique (but I agree that the explanation in that notebook is somewhat poor). ...
user avatar
3 votes

Learning computational science through guided discovery

I learn computation science through Practical Numerical Methods with Python. https://github.com/numerical-mooc/numerical-mooc/wiki It covers finite differencing and many other numerical algorithms. ...
user avatar
3 votes

Interpolation of Data Value using Optimized Weighting of Its Features

The problem is not convex, as one can verify using the simplest possible choice: $N=2, d=1$ and with data $x_1=x_2=1, f_1=1, f_2=0$. In that case, your objective function is $$ F(C) = 2\left(1-\exp(-...
user avatar
3 votes

Seeking a free symbolic regression software

I found the gramEvol R package flexible and easy to use. They have a small tutorial in which they rederive Kepler's third law from data. Note that it relies on Genetic Programmic for its optimisation ...
user avatar
3 votes
Accepted

Why is FLOP(Floating Point Operations Per Second) mentioned as a specification on every GPU?

This metric is pretty much as misleading (or useful, depending on your perspective) for GPUs as it is for CPUs. Currently, a lot of applications/algorithm's implementations are limited more by memory ...
user avatar
  • 8,382
3 votes
Accepted

Interpolation vs. Neural network

I have limited experience in machine learning; however, simplifying, you can think of it as a "trained black box". I would say, you have to know a lot about your problem, the behaviour of your ...
user avatar
  • 8,382
3 votes

How to filter customer voice from customer - agent conversation recordings?

This seems to be the "Cocktail Party Problem". Andrew Ng's machine learning course on Coursera gives a solution based on SVD for this problem. See the first week's course notes. Ng refers to ...
user avatar
  • 428
2 votes

Seeking a free symbolic regression software

There is also a package for R called rgp. Visit this link. https://cran.r-project.org/web/packages/rgp/index.html I have not used rgp as I have only begun to use R seriously but it seemed like a ...
user avatar
  • 21
2 votes

fmincg implementation in Python

I assume you didn't specify the fprime parameter. If you don't provide this param fmin_cg has to figure out its own solution ...
user avatar
2 votes
Accepted

How to train a model to classify object trajectories?

Just some quick ideas from someone who works in the realm of feature extraction in physical systems modeling such as this: you want to find the simplest and strongest differentiating characteristics ...
user avatar
  • 201
2 votes

Correct Weights Neural Network

One thing you should keep in mind, for future reference, is when you go to find the weights for an Artificial Neural Network using the Backpropogation algorithm with Gradient Descent (as you are), you'...
user avatar
  • 3,673
2 votes

Feedforward net lagged prediction

The reason it looks like it's just shifted is an illusion. One step prediction plots can be very deceiving for the unseasoned. Basically what's happening is that your time series is not very ...
user avatar
  • 870

Only top scored, non community-wiki answers of a minimum length are eligible