5

So, you want to invert your matrix $A=\Phi^T\Phi$. For $A$ to be invertible it must not have zero eigenvalues. We can show that $A$ is positive semi-definite as follows. Positive semi-definite means that the eigenvalues of $A$ are $\geq 0$. This is equivalent to showing $y^TAy \geq 0, \forall y \neq 0 $. $$ y^TAy = y^T\Phi^T\Phi{y}=(\Phi{y})^T(\Phi{y}) \geq ...


5

So bottom line is I don't see any comprehensive work on the use of AI in M&S as a whole, let's say having models that can learn how to produce new improved models using the existing models. There's definitely some work out there on this. This is the field of scientific machine learning. Currently there's three major paths that I'd break it into: Neural ...


4

The general idea that you have of learning an easy to compute model from results of your detailed simulation model and then optimizing the easy to compute model is long-established. The easy to compute model is typically called a surrogate model or a response surface model. Once the surrogate is available, you can use conventional optimization techniques ...


4

Look up something on Tikhonov regularization, also known as ridge regression in machine learning. This is a standard technique (but I agree that the explanation in that notebook is somewhat poor). Technically speaking, it does not affect the numerical stability of that algorithm, but it modifies the problem to a more well-conditioned one, from $\min \|\Phi \...


4

Think of the simplest case when $\Phi$ is a scalar value. Not well defined: $$ \boldsymbol \theta^\text{ML} = (0^T 0)^{-1}0^T ~ y = \frac{1}{0} 0~y= \frac{0}{0} $$ Well defined: $$ \boldsymbol \theta^\text{ML} = (0^T 0 + \kappa)^{-1}0^T~y =\frac{1}{\kappa} 0 ~y= 0 $$


3

This seems to be the "Cocktail Party Problem". Andrew Ng's machine learning course on Coursera gives a solution based on SVD for this problem. See the first week's course notes. Ng refers to Sam Roweis, Yair Weiss & Eero Simoncelli but I can't seem to find the reference on Google Scholar.


2

Most likely, you have the problem set up correctly and just need to adjust various things. What is the scale for altitude? You probably want to normalize it if you haven't already, especially since it seems like the fault parameter and mach are both $\approx$[0,1] 100 units is quite a lot. Note that more units =/= lower error, especially for something the ...


2

50 is a lot of parameters. You could try doing a basic first order sensitivity analysis to determine whether you can drop any of these. Using Bayesian Optimization to minimize a cost function is one way of dealing with the problem you've encountered. But remember that your standard L2 norm might have counterintuitive behaviours in high dimensions (see On the ...


1

All processors have counters that can be used to count all sorts of things between a point A in your program, and a point B. Examples are the number of floating point operations performed, the number of branches encountered, the number of cache misses, etc. I don't know, of course, what the authors of the paper you quote did, but it's not very difficult to ...


1

I'll start with a disclaimer, my PhD is in the fast computation of eigenvalues, my specialty is not in machine learning at all. This is just some stuff I remember from some master level courses. I have two ideas that might work. Idea 1 Traditional convolutional neural nets are very good at classifying. For example, "does this image contain a dog", ...


1

Since you are interested in ResNet, you may want to check out this repo: https://github.com/steffi7574/LayerParallelLearning It is based on the idea of "parallel-in-layer" and uses XBraid to distribute the layers. It is not exactly PETSc or Trillinos, but it is close. I have looked into distributed learning, more specifically, model parallelism, ...


1

I'm biased here, but my colleague Edward Chuah has done some research on identifying and predicting these kinds of failures, though we have not taken full advantage of these predictions, yet, at TACC. Perhaps, also, there's not as much ML in this as you'd like to see, depending on your definition of ML.


Only top voted, non community-wiki answers of a minimum length are eligible