Skip to main content
Share Your Experience: Take the 2024 Developer Survey

Questions tagged [machine-learning]

Machine learning, a branch of artificial intelligence, is the science of getting computer systems to meaningfully act without being explicitly programmed by human.

Filter by
Sorted by
Tagged with
0 votes
0 answers
26 views

OpenFOAM API: Access constructed matrix

I am interested in leveraging OpenFOAM to conduct data assimilation studies using 4DVar. I already have CFD data, and I have synthetic measurement data. All I need to be able to do is to call OpenFOAM ...
user3166083's user avatar
2 votes
0 answers
64 views

Which class of PDEs is well-suited for multistep methods?

I am experimenting with machine learning techniques to solve partial differential equations (PDEs). My goal is to use solutions from previous time steps to predict the solution at the next time step, ...
user572780's user avatar
0 votes
0 answers
10 views

Is the following the correct implementation of VGG network?

As exercise I am implementing few fundamental networks. Specifically right now I am implementing VGG The code I've got at the moment is the following: class MyVGG(nn.Module): ...
user8469759's user avatar
10 votes
1 answer
335 views

Is it possible to express an arbitrary tensor contraction in terms of BLAS routines?

I noticed that libraries like numpy and pytorch are able to perform arbitrary tensor contractions at speeds similar to comparably sized matrix multiplications. This leads me to believe that underneath ...
ilya's user avatar
  • 111
0 votes
0 answers
22 views

dual svm square hinge loss

Let $x_1,\dots,x_n\in \mathbb{R}^n$, $y_1,\dots,y_n\in \{-1,1\}$, $\lambda \ge 0$ and $K$ be the invertible Gram matrix $K=(x_i\cdot x_j)_{ij}$. Consider $$ (P) \qquad \qquad \min_{a\in \mathbb{R}^n} \...
Smilia's user avatar
  • 468
0 votes
0 answers
50 views

Nanograd differentation; what is going inside the python code

I am reading up on deep learning and I am trying to understand the backpropagation methods in python nanograd. See; https://github.com/rasmusbergpalm/nanograd This is a method for computing the ...
economist101's user avatar
0 votes
0 answers
41 views

Verification of a Function Definition in Python

I want to write a function $f$ and it is defined as $f = - \nabla \cdot(|\nabla u|^{p-2} \nabla u) $ and I exact solution $u(x) = \tilde{u}(r) = 1 - \frac{p-1}{p-2} \left( s^{p/{p-1}} - (1-s)^{p/{p-1}}...
User124356's user avatar
1 vote
0 answers
74 views

What architecture of CNN to solving a image regression problem (case study: solving Poisson equation)?

I've been working on solving Poisson problem using CNN model (you can ignore the Poisson problem part if you not familiar and jump to the image processing/CNN part). More specifically, I am solving ...
samueljohlal's user avatar
0 votes
1 answer
59 views

The row loss gradients

Suppose the original loss function is $$\min_{\mathbf{V}}\frac{1}{2}\|\mathbf{V} - Q(\mathbf{V})\odot\mathbf{U}\mathbf{E} - \beta Q(\mathbf{V})\mathbf{V}\|_2^2$$ where $\odot$ denotes the element-wise ...
Zuba Tupaki's user avatar
0 votes
1 answer
52 views

Dimensionality reduction between discrete wavelet families

I have what it may be a ridiculous question (since I don't know much about wavelets), but here I go. I am using different Discrete Wavelet families to extract texture features from images. I plan to ...
PPM's user avatar
  • 3
12 votes
2 answers
1k views

Faster Logistic Function

I've noticed that a fairly significant number of cycles in one of my programs are being consumed by the logistic function: $$f(x)=\frac{1}{1+e^{-x}}$$ Is there a good approximation I can use to reduce ...
Richard's user avatar
  • 3,971
3 votes
1 answer
326 views

What problems does softmax() solve and when should I think of using it - in simple terms

I just for the first time saw the function softmax() in this SO answer to How do I use a minimization function in scipy with constraints and was intrigued. Another way of weighting variables where ...
uhoh's user avatar
  • 1,058
0 votes
1 answer
88 views

Does the loss function in a deep neural network act as a norm?

I read somewhere that the Measn squared error loss function acts as L2 norm of the paramter vector. I would like to know if I am using binary cross entropy loss function, do I need to calculate the ...
Jeet's user avatar
  • 113
4 votes
0 answers
194 views

SINDy Vs standard methods for system identification

I have been trying to understand the recently proposed Sparse Identification of Nonlinear Dynamics SINDy. Despite several attempts, I seem to fail to understand the difference between SINDy and the ...
Chenna K's user avatar
  • 944
14 votes
3 answers
2k views

Why aren't Krylov subspace methods popular in the Machine Learning community compared to Gradient Descent?

Historically, iterative methods for solving relatively simple-structured systems $Ax=b$ with $A$ being a $4\times 4$ matrix or to find the eigenvalues of that matrix assuming in both problems that $A$ ...
SPARSE's user avatar
  • 169
2 votes
1 answer
177 views

Variational loss of hp-Variational Physics Informed Neural Networks for 2D-Poisson Equation in Tensorflow

I am trying to reproduce the results from the hp-VPINN paper (https://arxiv.org/pdf/2003.05385.pdf) on tensorflow (v1) for Poisson's equation, particularly the two-dimensional Poisson equation. In one ...
M.V.'s user avatar
  • 21
4 votes
0 answers
93 views

Identifying an unknown P.D.E. from solution data

I have a black-box simulation that produces the time evolution of a probability density function p(x, t) in 1 dimension from arbitrary initial conditions p(x, 0). The underlying simulation occurs on a ...
beables's user avatar
  • 41
10 votes
1 answer
1k views

Sensitivity of $y$ w.r.t. to $x$ in $y=f(x)$ where f is a routine

Given a model $f$ as a programming routine, such that we are able to compute $y=f(x)$ for any $x \in \mathcal{D}$, I am interested in the sensitivity (or let us say derivative) of $y$ with respect to $...
outlaw's user avatar
  • 264
0 votes
0 answers
141 views

Getting euclidean distance between vector A and C without anyway of retrieving them when their distances with a common vector B is known

Motivation: My plan is to get the overall euclidean distance matrix for all the vectors in N number of dataset. Each dataset is basically an array of n-dimensional points. For e.g: A dataset can be ...
Shihab Ullah's user avatar
0 votes
1 answer
238 views

What are the benefits of using machine learning for interpolation over traditional interpolation methods?

I am trying to get a better understanding of the application of function approximation with machine learning. My question is simple, how does function approximation with ML compare to traditional ...
Frosty's user avatar
  • 27
1 vote
1 answer
200 views

Improving function approximation with neural network

I am building a neural network to approximate a data set which takes 3 inputs and gives 1 output. After testing the network using a few different iterations of hidden layers and adjusting optimizers ...
Frosty's user avatar
  • 27
1 vote
0 answers
55 views

Neural networks trained with subspace iteration algorithms for solving SCF eigenvalue problems

I am toying with the idea of using current libraries available with SCF (Self-Consistent Field) subspace iteration codes to train an ANN (Artificial Neural Network) to solve for SCF eigenvalue ...
Jonathan's user avatar
  • 111
5 votes
3 answers
517 views

Advantage of diagonal "jitter" for numerical stability?

In a machine learning code, that computes optimum parameters $\theta _{MLE}$ of a linear regression model, by maximum likelihood estimation: $$ \boldsymbol \theta^\text{ML} = (\boldsymbol\Phi^T\...
Algo's user avatar
  • 304
2 votes
1 answer
72 views

How to filter customer voice from customer - agent conversation recordings?

I have a doubt on the project I'm working now. Actually I want only customer voice from the recordings which contains customer-agent conversation.But I have no idea to filter customer voice from ...
sowmiya's user avatar
  • 21
1 vote
1 answer
1k views

How to calculate the number of floating point operations a task/ process requires? (not FLOP/s, but FLOP)

There have been many papers quoting FLOP to quote the performance of a specific approach in machine learning. For example, We trained two models with different capacities: BlazePose Full (6.9 MFlop, ...
Ben Butterworth's user avatar
2 votes
1 answer
204 views

Is there a way to return a substring of a string using Convolutional Neural Networks?

I'm a PhD student in genetics and molecular biology working on an algorithm to identify if a DNA sequence is either a transposable element (TE) or not a TE using convolutional neural networks, and it'...
Tiago Minuzzi's user avatar
3 votes
2 answers
849 views

Optimization of expensive model with many parameters

I have a physical model which takes $\sim50$ parameters and gives $\sim2000$ outputs taking tens of minutes to run. I need to optimize these parameters to give outputs as close as possible to data. ...
Ghorbalchov's user avatar
1 vote
0 answers
33 views

How property-invariance is imposed to neural nets?

I was wondering how specific symmetries or constraints such as property-invariance transformation are imposed on any (deep) neural net when they are trained. I'll appreciate it if anyone can aware me ...
arash's user avatar
  • 21
1 vote
1 answer
188 views

Artificial Intelligence, Modeling and Simulation

Artificial Intelligence (AI) and its subsets i.e. Deep Learning (DL), Machine Learning (ML) etc. are becoming more and more ubiquitous in engineering, technology and science. Modeling and Simulation (...
Dude's user avatar
  • 580
2 votes
1 answer
142 views

Machine Learning for Optimization

I have a function which takes 100+ coefficients and outputs $x$. I wish to optimise $x$. Running the simulation 50 000 times will take around 15 minutes, however, this happens in parallel - and the ...
Tomi's user avatar
  • 123
0 votes
0 answers
54 views

Unsupervised event prediction with time series data

I have to solve a problem using unsupervised time series forecast. I have a dataset where three columns are given, 1. Endtime: when the row was generated ( it generates a row after each 54-55 seconds)...
ujjwal anand's user avatar
-1 votes
2 answers
75 views

How to find the relationship between an independent variable in a time series and a single dependent variable

I have a dataset of crop yield of a seasonal crop, under environmental conditions (rainfall, humidity, temperature etc.). Daily environmental conditions are recorded over few years with the crop yield ...
Dulanga Kithmini's user avatar
1 vote
0 answers
34 views

Why is scoring a separate problem even after docking is done?

With regards to protein-ligand interactions, we talk about two concepts, docking and scoring. I understand that docking means to find the best orientation of a ligand at the active site of a protein, ...
Mahathi Vempati's user avatar
0 votes
0 answers
97 views

exploding gradients in gradient descent procedure of multi-output ridge regression

Multi-output ridge regression: $$W^{*}=\underset{W}{\arg \min } \frac{1}{\mathcal{N}}\|Y-WX\|_{F}^{2}+\lambda\|W\|_{F}^{2}$$ There are $Q$ outputs, $N$ samples, and $P$ covariates (features). $\hat{...
venkat's user avatar
  • 21
2 votes
0 answers
203 views

Proving convexity of Frobenius norm and correlation function formulations of an optimization problem

I have been working on formulating my requirements in the form of an optimization problem in a multi-output regression setting. Firstly, I would like to make the variables I used in the problem and ...
venkat's user avatar
  • 21
13 votes
1 answer
9k views

When training a neural network, why choose Adam over L-BGFS for the optimizer?

More specifically, when training a neural network, what reasons are there for choosing an optimizer from the family consisting of stochastic gradient descent (SGD) and its extensions (RMSProp, Adam, ...
tietäjä's user avatar
  • 133
5 votes
3 answers
316 views

Deep learning using Distributed linear algebra

Is there any deep learning library based on Trillinos or Petsc linear algebra?
computational_scientist's user avatar
1 vote
0 answers
91 views

Eigenfaces Algorithm

This might be a silly quesntion but recently I've been trying to program the eigenface algorithm using PCA, so I arranged the face vectors vertically in a matrix X such as: X = [x1,x2,x3,...,xn]; In ...
Marcus's user avatar
  • 31
2 votes
1 answer
27 views

How To Interpret PCA Points Labeled With Specific Data Dimensions

I've done some PCA on my own, and am familiar with the basic concepts of how PCA components are calculated and applied. However, I'm working on a research project and am confused as to how to ...
Zachary Rohman's user avatar
3 votes
0 answers
119 views

A maximization problem, with motivation in machine learning

Consider the minimization problem described this paper. Let $f_{\lambda}$ be the minimizer. As a part of extending my work, I am able to show the following facts $$\lim_\limits{\lambda \to 0}\|f_{\...
Rajesh D's user avatar
  • 141
1 vote
0 answers
22 views

Tracking channel states using Machine Learning

I am new in AI and would like to apply machine learning to estimate the channel states. I have a set of data. It is a matrix of 10000*8. Each row of this matrix is regarding a time step, i.e., 1st row ...
Susan's user avatar
  • 33
2 votes
1 answer
73 views

Activation function with special conditions in machine learning

I only have a basic understanding of deep learning, but looking through it I had an idea on how to approximate global minima of the NN. However, for it's activation function I am only able to use: ...
olukatorzu's user avatar
3 votes
2 answers
2k views

Interpolation vs. Neural network

I am seeking knowledge from the community. I am solving a transport PDE (conservation of solute mass) using COMSOL. At each Newton-Raphson iteration, I need to update a constant called $Kd$ for some ...
Daniel's user avatar
  • 99
0 votes
1 answer
64 views

Training accuracy improves but test set accuracy remains the same

I have built an ANN model with 5 hidden layers and 100 nodes in each layer to solve a multilabel classification problem. After the first run, I get a training accuracy of ~66% and a test set accuracy ...
Chayan Chatterjee's user avatar
1 vote
0 answers
54 views

Singular Spectrum Analysis Explanation

I need you to help me understand the Singular Spectrum Analysis algorithm. I already read a lot of articles about the subject but they never answered my questions like what is the mathematical reason ...
Olivier's user avatar
  • 11
1 vote
0 answers
110 views

Connection between piecewise linear basis functions and RELU activation function

ReLU activation is defined as follows $$\sigma(x)=\max(0, x).$$ Let's assume that I have deep network of 1 hidden layer, than output from my layer has form $$ f(x)= \sigma(Wx +b), $$ where matrix W ...
computational_scientist's user avatar
1 vote
1 answer
35 views

Want to make sense of array dimensions in logistic regression algorithms

I am trying to implement a simple logistic regression algorithm from scratch in python (for learning purposes). Every article I've seen online so far presents the following expression for $z$ (...
user32882's user avatar
  • 251
2 votes
1 answer
142 views

Artificial neural networks for Temperature prediction

Imagine I want to consider the temperature for a process given several input varibales. The temperature can be anywhere between 400 and 500 K. Consider I have experimental data to train the network ...
Gesetzt's user avatar
  • 33
3 votes
0 answers
108 views

Are the No Free Lunch Theorems Useful for Anything?

I have been thinking about the No Free Lunch (NFL) theorems lately, and I have a question which probably every one who has ever thought of the NFL theorems has also had. I am asking this question here,...
Surgical Commander's user avatar
1 vote
3 answers
288 views

Optimization of a blackbox function

Let's say that we have an objective function $f(\mathbf x,\mathbf y)$ which has the parameters $\mathbf x=[x_1\ldots x_n]$ and $\mathbf y=[y_1\ldots y_n]$. Here, $\mathbf y$ is a blackbox variable ...
Janson 7's user avatar