Questions tagged [machine-learning]
Machine learning, a branch of artificial intelligence, is the science of getting computer systems to meaningfully act without being explicitly programmed by human.
120
questions
0
votes
0
answers
45
views
Nanograd differentation; what is going inside the python code
I am reading up on deep learning and I am trying to understand the backpropagation methods in python nanograd.
See; https://github.com/rasmusbergpalm/nanograd
This is a method for computing the ...
0
votes
0
answers
40
views
Verification of a Function Definition in Python
I want to write a function $f$ and it is defined as $f = - \nabla \cdot(|\nabla u|^{p-2} \nabla u) $ and I exact solution $u(x) = \tilde{u}(r) = 1 - \frac{p-1}{p-2} \left( s^{p/{p-1}} - (1-s)^{p/{p-1}}...
- 143
1
vote
0
answers
71
views
What architecture of CNN to solving a image regression problem (case study: solving Poisson equation)?
I've been working on solving Poisson problem using CNN model (you can ignore the Poisson problem part if you not familiar and jump to the image processing/CNN part). More specifically, I am solving ...
- 11
0
votes
1
answer
56
views
The row loss gradients
Suppose the original loss function is
$$\min_{\mathbf{V}}\frac{1}{2}\|\mathbf{V} - Q(\mathbf{V})\odot\mathbf{U}\mathbf{E} - \beta Q(\mathbf{V})\mathbf{V}\|_2^2$$
where $\odot$ denotes the element-wise ...
- 55
0
votes
1
answer
40
views
Dimensionality reduction between discrete wavelet families
I have what it may be a ridiculous question (since I don't know much about wavelets), but here I go.
I am using different Discrete Wavelet families to extract texture features from images. I plan to ...
- 3
0
votes
0
answers
12
views
Non-Temporal Weighted Graph Datasets
I am searching for datasets to evaluate an algorithm designed for tasks such as node-classification (edge-prediction, etc.) on weighted and potentially directed graphs.
The Stanford Network Analysis ...
- 1
12
votes
2
answers
989
views
Faster Logistic Function
I've noticed that a fairly significant number of cycles in one of my programs are being consumed by the logistic function:
$$f(x)=\frac{1}{1+e^{-x}}$$
Is there a good approximation I can use to reduce ...
- 3,921
3
votes
1
answer
275
views
What problems does softmax() solve and when should I think of using it - in simple terms
I just for the first time saw the function softmax() in this SO answer to How do I use a minimization function in scipy with constraints and was intrigued.
Another way of weighting variables where ...
- 1,026
0
votes
1
answer
81
views
Does the loss function in a deep neural network act as a norm?
I read somewhere that the Measn squared error loss function acts as L2 norm of the paramter vector. I would like to know if I am using binary cross entropy loss function, do I need to calculate the ...
- 113
4
votes
0
answers
143
views
SINDy Vs standard methods for system identification
I have been trying to understand the recently proposed Sparse Identification of Nonlinear Dynamics SINDy. Despite several attempts, I seem to fail to understand the difference between SINDy and the ...
- 875
14
votes
3
answers
2k
views
Why aren't Krylov subspace methods popular in the Machine Learning community compared to Gradient Descent?
Historically, iterative methods for solving relatively simple-structured systems $Ax=b$ with $A$ being a $4\times 4$ matrix or to find the eigenvalues of that matrix assuming in both problems that $A$ ...
- 169
2
votes
1
answer
174
views
Variational loss of hp-Variational Physics Informed Neural Networks for 2D-Poisson Equation in Tensorflow
I am trying to reproduce the results from the hp-VPINN paper (https://arxiv.org/pdf/2003.05385.pdf) on tensorflow (v1) for Poisson's equation, particularly the two-dimensional Poisson equation.
In one ...
- 21
4
votes
0
answers
90
views
Identifying an unknown P.D.E. from solution data
I have a black-box simulation that produces the time evolution of a probability density function p(x, t) in 1 dimension from arbitrary initial conditions p(x, 0). The underlying simulation occurs on a ...
- 41
10
votes
1
answer
1k
views
Sensitivity of $y$ w.r.t. to $x$ in $y=f(x)$ where f is a routine
Given a model $f$ as a programming routine, such that we are able to compute $y=f(x)$ for any $x \in \mathcal{D}$, I am interested in the sensitivity (or let us say derivative) of $y$ with respect to $...
- 264
0
votes
0
answers
141
views
Getting euclidean distance between vector A and C without anyway of retrieving them when their distances with a common vector B is known
Motivation:
My plan is to get the overall euclidean distance matrix for all the vectors in N number of dataset. Each dataset is basically an array of n-dimensional points. For e.g: A dataset can be ...
- 11
0
votes
1
answer
216
views
What are the benefits of using machine learning for interpolation over traditional interpolation methods?
I am trying to get a better understanding of the application of function approximation with machine learning. My question is simple, how does function approximation with ML compare to traditional ...
- 27
1
vote
1
answer
196
views
Improving function approximation with neural network
I am building a neural network to approximate a data set which takes 3 inputs and gives 1 output. After testing the network using a few different iterations of hidden layers and adjusting optimizers ...
- 27
1
vote
0
answers
54
views
Neural networks trained with subspace iteration algorithms for solving SCF eigenvalue problems
I am toying with the idea of using current libraries available with SCF (Self-Consistent Field) subspace iteration codes to train an ANN (Artificial Neural Network) to solve for SCF eigenvalue ...
- 111
5
votes
3
answers
453
views
Advantage of diagonal "jitter" for numerical stability?
In a machine learning code, that computes optimum parameters $\theta _{MLE}$ of a linear regression model, by maximum likelihood estimation:
$$ \boldsymbol \theta^\text{ML} = (\boldsymbol\Phi^T\...
- 304
2
votes
1
answer
67
views
How to filter customer voice from customer - agent conversation recordings?
I have a doubt on the project I'm working now.
Actually I want only customer voice from the recordings which contains customer-agent conversation.But I have no idea to filter customer voice from ...
- 21
1
vote
1
answer
1k
views
How to calculate the number of floating point operations a task/ process requires? (not FLOP/s, but FLOP)
There have been many papers quoting FLOP to quote the performance of a specific approach in machine learning. For example,
We trained two models with different capacities: BlazePose Full (6.9 MFlop, ...
- 111
2
votes
1
answer
189
views
Is there a way to return a substring of a string using Convolutional Neural Networks?
I'm a PhD student in genetics and molecular biology working on an algorithm to identify if a DNA sequence is either a transposable element (TE) or not a TE using convolutional neural networks, and it'...
3
votes
2
answers
573
views
Optimization of expensive model with many parameters
I have a physical model which takes $\sim50$ parameters and gives $\sim2000$ outputs taking tens of minutes to run. I need to optimize these parameters to give outputs as close as possible to data. ...
- 131
1
vote
0
answers
31
views
How property-invariance is imposed to neural nets?
I was wondering how specific symmetries or constraints such as property-invariance transformation are imposed on any (deep) neural net when they are trained.
I'll appreciate it if anyone can aware me ...
- 21
1
vote
1
answer
163
views
Artificial Intelligence, Modeling and Simulation
Artificial Intelligence (AI) and its subsets i.e. Deep Learning (DL), Machine Learning (ML) etc. are becoming more and more ubiquitous in engineering, technology and science. Modeling and Simulation (...
- 570
2
votes
1
answer
139
views
Machine Learning for Optimization
I have a function which takes 100+ coefficients and outputs $x$. I wish to optimise $x$.
Running the simulation 50 000 times will take around 15 minutes, however, this happens in parallel - and the ...
- 123
0
votes
0
answers
54
views
Unsupervised event prediction with time series data
I have to solve a problem using unsupervised time series forecast. I have a dataset where three columns are given,
1. Endtime: when the row was generated ( it generates a row after each 54-55 seconds)...
-1
votes
2
answers
74
views
How to find the relationship between an independent variable in a time series and a single dependent variable
I have a dataset of crop yield of a seasonal crop, under environmental conditions (rainfall, humidity, temperature etc.). Daily environmental conditions are recorded over few years with the crop yield ...
1
vote
0
answers
32
views
Why is scoring a separate problem even after docking is done?
With regards to protein-ligand interactions, we talk about two concepts, docking and scoring. I understand that docking means to find the best orientation of a ligand at the active site of a protein, ...
- 111
0
votes
0
answers
90
views
exploding gradients in gradient descent procedure of multi-output ridge regression
Multi-output ridge regression:
$$W^{*}=\underset{W}{\arg \min } \frac{1}{\mathcal{N}}\|Y-WX\|_{F}^{2}+\lambda\|W\|_{F}^{2}$$
There are $Q$ outputs, $N$ samples, and $P$ covariates (features).
$\hat{...
- 21
2
votes
0
answers
186
views
Proving convexity of Frobenius norm and correlation function formulations of an optimization problem
I have been working on formulating my requirements in the form of an optimization problem in a multi-output regression setting.
Firstly, I would like to make the variables I used in the problem and ...
- 21
13
votes
1
answer
8k
views
When training a neural network, why choose Adam over L-BGFS for the optimizer?
More specifically, when training a neural network, what reasons are there for choosing an optimizer from the family consisting of stochastic gradient descent (SGD) and its extensions (RMSProp, Adam, ...
- 133
5
votes
3
answers
313
views
Deep learning using Distributed linear algebra
Is there any deep learning library based on Trillinos or Petsc linear algebra?
1
vote
0
answers
91
views
Eigenfaces Algorithm
This might be a silly quesntion but recently I've been trying to program the eigenface algorithm using PCA, so I arranged the face vectors vertically in a matrix X such as: X = [x1,x2,x3,...,xn];
In ...
- 31
2
votes
1
answer
26
views
How To Interpret PCA Points Labeled With Specific Data Dimensions
I've done some PCA on my own, and am familiar with the basic concepts of how PCA components are calculated and applied. However, I'm working on a research project and am confused as to how to ...
3
votes
0
answers
119
views
A maximization problem, with motivation in machine learning
Consider the minimization problem described this paper. Let $f_{\lambda}$ be the minimizer. As a part of extending my work, I am able to show the following facts
$$\lim_\limits{\lambda \to 0}\|f_{\...
- 141
1
vote
0
answers
22
views
Tracking channel states using Machine Learning
I am new in AI and would like to apply machine learning to estimate the channel states.
I have a set of data. It is a matrix of 10000*8. Each row of this matrix is regarding a time step, i.e., 1st row ...
- 33
2
votes
1
answer
73
views
Activation function with special conditions in machine learning
I only have a basic understanding of deep learning, but looking through it I had an idea on how to approximate global minima of the NN.
However, for it's activation function I am only able to use:
...
- 21
3
votes
2
answers
1k
views
Interpolation vs. Neural network
I am seeking knowledge from the community. I am solving a transport PDE (conservation of solute mass) using COMSOL. At each Newton-Raphson iteration, I need to update a constant called $Kd$ for some ...
- 99
0
votes
1
answer
63
views
Training accuracy improves but test set accuracy remains the same
I have built an ANN model with 5 hidden layers and 100 nodes in each layer to solve a multilabel classification problem. After the first run, I get a training accuracy of ~66% and a test set accuracy ...
1
vote
0
answers
54
views
Singular Spectrum Analysis Explanation
I need you to help me understand the Singular Spectrum Analysis algorithm. I already read a lot of articles about the subject but they never answered my questions like what is the mathematical reason ...
- 11
1
vote
0
answers
110
views
Connection between piecewise linear basis functions and RELU activation function
ReLU activation is defined as follows
$$\sigma(x)=\max(0, x).$$
Let's assume that I have deep network of 1 hidden layer, than output from my layer has form
$$ f(x)= \sigma(Wx +b), $$
where matrix W ...
1
vote
1
answer
34
views
Want to make sense of array dimensions in logistic regression algorithms
I am trying to implement a simple logistic regression algorithm from scratch in python (for learning purposes). Every article I've seen online so far presents the following expression for $z$ (...
- 251
2
votes
1
answer
140
views
Artificial neural networks for Temperature prediction
Imagine I want to consider the temperature for a process given several input varibales. The temperature can be anywhere between 400 and 500 K. Consider I have experimental data to train the network ...
- 33
3
votes
0
answers
108
views
Are the No Free Lunch Theorems Useful for Anything?
I have been thinking about the No Free Lunch (NFL) theorems lately, and I have a question which probably every one who has ever thought of the NFL theorems has also had. I am asking this question here,...
1
vote
3
answers
283
views
Optimization of a blackbox function
Let's say that we have an objective function $f(\mathbf x,\mathbf y)$ which has the parameters $\mathbf x=[x_1\ldots x_n]$ and $\mathbf y=[y_1\ldots y_n]$. Here, $\mathbf y$ is a blackbox variable ...
- 81
1
vote
0
answers
49
views
What is a performant clustering algorithm for approx 10,000 vectors of approx 30 dimension?
I have a set of real-valed vectors, for example
$S = \{v_1, v_2, ..., v_k\}$
$v_i = \begin{pmatrix} age_i \\ height_i \\ weight_i \\ ... \end{pmatrix}$
or whatever. Each vector has on the order of ...
- 111
1
vote
0
answers
100
views
Is there any ML solutions for permutations?
In Machine Learning field, is there any work done, or research, or info in general about finding optimal permutations?
For eg. to find a number of optimal arrangements of sliced shapes on 2D sheet of ...
- 111
1
vote
1
answer
337
views
Why is FLOP(Floating Point Operations Per Second) mentioned as a specification on every GPU?
Counting FLOP may not be representative of the actual algorithm real world performance but still all the GPU manufacturers mention FLOPS as a metric of Performance on GPU. Is there any way that this ...
- 113
4
votes
1
answer
164
views
Using the PAST algorithm to find eigenvectors
I'm working on trying to extract the eigenvectors from a series of observations from a random variable, by using the PAST algorithm, see e.g. 6.2.3 in this book:
Large pdf.
I don't understand the ...
- 41