Skip to main content
OverflowAI is here! AI power for your Stack Overflow for Teams knowledge community. Learn more

Questions tagged [machine-learning]

Machine learning, a branch of artificial intelligence, is the science of getting computer systems to meaningfully act without being explicitly programmed by human.

Filter by
Sorted by
Tagged with
16 votes
5 answers
5k views

Apply PCA on very large sparse matrix

I am doing a text classification task with R, and I obtain a document-term matrix with size 22490 by 120,000 (only 4 million non-zero entries, less than 1% entries). Now I want to reduce the ...
Ensom Hodder's user avatar
16 votes
3 answers
4k views

Python OSS alternatives for Matlab Neural Network Toolbox. Any intercomparisons?

I'd like to be independent of commercial software for my scientific work. I find a dependence an commercial packages such as Matlab and its toolboxes unsatisfactory, because I do not know if I will ...
gerrit's user avatar
  • 260
14 votes
3 answers
2k views

Why aren't Krylov subspace methods popular in the Machine Learning community compared to Gradient Descent?

Historically, iterative methods for solving relatively simple-structured systems $Ax=b$ with $A$ being a $4\times 4$ matrix or to find the eigenvalues of that matrix assuming in both problems that $A$ ...
SPARSE's user avatar
  • 169
13 votes
6 answers
5k views

Seeking a free symbolic regression software

Now that Formulize / Eureqa started charging $2500 a year for using it and having crippled the trial version, does anyone know of any replacements that can do similar things like find an equation ...
Rick T's user avatar
  • 231
13 votes
1 answer
9k views

When training a neural network, why choose Adam over L-BGFS for the optimizer?

More specifically, when training a neural network, what reasons are there for choosing an optimizer from the family consisting of stochastic gradient descent (SGD) and its extensions (RMSProp, Adam, ...
tietäjä's user avatar
  • 133
12 votes
2 answers
1k views

Faster Logistic Function

I've noticed that a fairly significant number of cycles in one of my programs are being consumed by the logistic function: $$f(x)=\frac{1}{1+e^{-x}}$$ Is there a good approximation I can use to reduce ...
Richard's user avatar
  • 3,971
11 votes
2 answers
8k views

Use of machine learning in computational fluid dynamics

Background: I have only built one working numeric solution to 2d Navier-Stokes, for a course. It was a solution for lid-driven cavity flow. The course, however, discussed a number of schemas for ...
EngrStudent's user avatar
10 votes
1 answer
1k views

Sensitivity of $y$ w.r.t. to $x$ in $y=f(x)$ where f is a routine

Given a model $f$ as a programming routine, such that we are able to compute $y=f(x)$ for any $x \in \mathcal{D}$, I am interested in the sensitivity (or let us say derivative) of $y$ with respect to $...
outlaw's user avatar
  • 264
10 votes
2 answers
3k views

Do RBF kernel matrices tend to be ill-conditioned?

I use RBF kernel function to implement one kernel based machine learning algorithm(KLPP), the resulting kernel matrix $K$ $$K(i,j)= \exp\left({\frac{-(x_{i}-x_{j})^2}{ \sigma_{m}^2}}\right)$$ is ...
ZeyuHu's user avatar
  • 317
10 votes
2 answers
4k views

Calculating Lagrange coefficients for SVM in Python

I'm trying to write a full SVM implementation in Python and I have a few issues computing the Lagrange coefficients. First let me rephrase what I understand from the algorithm to make sure I'm on the ...
Charles Menguy's user avatar
10 votes
1 answer
334 views

Is it possible to express an arbitrary tensor contraction in terms of BLAS routines?

I noticed that libraries like numpy and pytorch are able to perform arbitrary tensor contractions at speeds similar to comparably sized matrix multiplications. This leads me to believe that underneath ...
ilya's user avatar
  • 111
9 votes
2 answers
187 views

Predict runtimes for dense linear algebra

I would like to predict runtimes for dense linear algebra operations on a specific architecture using a specific library. I would like to learn a model that approximates the function $F_{op} \;::\; $...
MRocklin's user avatar
  • 3,058
9 votes
3 answers
4k views

Are there tasks in machine learning which require double precision floating points?

Double-precision calculations are significantly slower or more expensive than single-precision calculations. For example, the NVidia Tesla which performs well on doubles is much more expensive then ...
Marat Zakirov's user avatar
9 votes
2 answers
4k views

Markov (Chain) image generators?

Markov Chains can be used to generate, or auto-complete, text. https://en.wikipedia.org/wiki/Markov_chain#Markov_text_generators Training text is read, and some information about the text is ...
alan2here's user avatar
  • 193
8 votes
3 answers
5k views

fmincg implementation in Python

I'm trying to re-implement Neural Networks in Python. I implemented the cost function and the backpropagation algorithm correctly. I have checked them by executing its Octave equivalent code. But ...
garak's user avatar
  • 223
7 votes
2 answers
244 views

Learning parameters of noise and filter coefficients from data where data and noise both have Gaussian distributions

Assume $X$ and $N$ are two sets of vectors (observations) from two different normal distributions, where $X$ represents clean data and $N$ represents noise; and $A$ a projection matrix of a filter. ...
PickleRick's user avatar
7 votes
4 answers
300 views

Testing for stability of a simulated dynamical system

Background and question I often work with simulations of dynamical systems and I usually track a single parameter $x$, such as the number of agents (for agents based models) or the error rate (for ...
Artem Kaznatcheev's user avatar
6 votes
1 answer
193 views

Support Vector Machines as Neural Nets?

This is more of a conceptual question. I have learned about Neural Nets, and I have some clue as to how Support Vector Machines work. I read somewhere however that given the appropriate kernel (is ...
Josh F's user avatar
  • 165
6 votes
2 answers
181 views

Books/Resources on Sparse Optimization?

I'm looking to learn more about Sparse Optimization and apply it to machine learning problems. Could you please recommend some books/resources on this topic? Both theoretical and applied are fine.
hattoriace's user avatar
5 votes
3 answers
510 views

Advantage of diagonal "jitter" for numerical stability?

In a machine learning code, that computes optimum parameters $\theta _{MLE}$ of a linear regression model, by maximum likelihood estimation: $$ \boldsymbol \theta^\text{ML} = (\boldsymbol\Phi^T\...
Algo's user avatar
  • 304
5 votes
4 answers
259 views

Learning computational science through guided discovery

I am currently trying to get through Pattern Classification by Duda et al (for a course). However, the book seems too dense for me. Pattern recognition seems like a topic that could be better learned ...
Avatrin's user avatar
  • 192
5 votes
1 answer
3k views

Logistic regression with Python

I am trying to code up logistic regression in Python using the SciPy fmin_bfgs function, but am running into some issues. I wrote functions for the logistic (...
tchakravarty's user avatar
5 votes
1 answer
133 views

Using an approximation algorithm to adapt parameter values of a given algorithm

Problem: I have an incremental online clustering algorithm which need 4 parameters that should be specified by the user before execution. The algorithm will gives "good results" if "a good parameter ...
user995434's user avatar
5 votes
3 answers
316 views

Deep learning using Distributed linear algebra

Is there any deep learning library based on Trillinos or Petsc linear algebra?
computational_scientist's user avatar
5 votes
1 answer
171 views

How to train a model to classify object trajectories?

I have a tracker that outputs the trajectory $(x,y,z)$ of an object (e.g., a can). I want to use these trajectories to train a classifier (i.e., SVM) in order to infer the activity that the person ...
Gianpiero Francesca's user avatar
5 votes
1 answer
153 views

Looking for an understandable discussion of creating Maximum Entropy classifiers

Texts, articles, and papers on Maximum Entropy Classifiers tend to come in two varieties: the more popular "upper level", and the more technical. The popular variety are good at explaining the ...
winwaed's user avatar
  • 658
4 votes
1 answer
4k views

Using scipy.optimize to implement a neural network with back propagation

My problem is something similar to this. I'm trying to implement a (Neural Network) Cost function, Back propogation algorithm in Python. The Neural Network has 3 layers. Hence 2 parameters to optimize ...
garak's user avatar
  • 223
4 votes
1 answer
164 views

Using the PAST algorithm to find eigenvectors

I'm working on trying to extract the eigenvectors from a series of observations from a random variable, by using the PAST algorithm, see e.g. 6.2.3 in this book: Large pdf. I don't understand the ...
knajp's user avatar
  • 41
4 votes
1 answer
1k views

Diffusion kernel "guide"

Diffusion kernels are kernels which "project" information about graphs into $R^n$ so that certain machine learning techniques can be performed. I have read through this paper and feel fairly ...
Xodarap's user avatar
  • 141
4 votes
0 answers
187 views

SINDy Vs standard methods for system identification

I have been trying to understand the recently proposed Sparse Identification of Nonlinear Dynamics SINDy. Despite several attempts, I seem to fail to understand the difference between SINDy and the ...
Chenna K's user avatar
  • 944
4 votes
0 answers
93 views

Identifying an unknown P.D.E. from solution data

I have a black-box simulation that produces the time evolution of a probability density function p(x, t) in 1 dimension from arbitrary initial conditions p(x, 0). The underlying simulation occurs on a ...
beables's user avatar
  • 41
4 votes
0 answers
81 views

Calculus of Variations with unknown cost function but some data

I have a problem that I've framed out in a particular way, but I don't know if I'm re-inventing the wheel here. Is there an existing literature base in this problem? Does it have a corresponding term ...
Sycorax's user avatar
  • 141
3 votes
2 answers
2k views

Interpolation vs. Neural network

I am seeking knowledge from the community. I am solving a transport PDE (conservation of solute mass) using COMSOL. At each Newton-Raphson iteration, I need to update a constant called $Kd$ for some ...
Daniel's user avatar
  • 99
3 votes
1 answer
2k views

Why am I not seeing faster neural network training after upgrading to a vastly better GPU?

I was previously running my neural networks using the Lasagne library to build and train neural networks in Theano on an NVIDIA GTX 750 Ti. I'm using a genetic algorithm to tune the hyperparameters of ...
Joels Elf's user avatar
  • 141
3 votes
2 answers
1k views

Applying same feature selection to multiple data sets with Weka

I am using the Weka workbench to train a protein fold classifier. I imported my training data into Weka and performed PCA-based feature selection. This seems to have worked fine, but now I cannot ...
Daniel Standage's user avatar
3 votes
1 answer
323 views

What problems does softmax() solve and when should I think of using it - in simple terms

I just for the first time saw the function softmax() in this SO answer to How do I use a minimization function in scipy with constraints and was intrigued. Another way of weighting variables where ...
uhoh's user avatar
  • 1,048
3 votes
1 answer
101 views

Improve optimization over 'mapping' of indices

I have two tables at my disposal, one work dataset and one reference dataset. Each dataset has got two columns, lets say these are fields A and B. I would like to associate the rows in the reference ...
kiriloff's user avatar
  • 343
3 votes
2 answers
833 views

Optimization of expensive model with many parameters

I have a physical model which takes $\sim50$ parameters and gives $\sim2000$ outputs taking tens of minutes to run. I need to optimize these parameters to give outputs as close as possible to data. ...
Ghorbalchov's user avatar
3 votes
1 answer
1k views

How to train an L2-regularized L1 Hinge Loss SVM using vowpal wabbit?

I'm trying to train an L2-regularized L1-hinge loss SVM using vowpal-wabbit. I use the following commands to train and test on the splice dataset: ...
Hugh Perkins's user avatar
3 votes
0 answers
119 views

A maximization problem, with motivation in machine learning

Consider the minimization problem described this paper. Let $f_{\lambda}$ be the minimizer. As a part of extending my work, I am able to show the following facts $$\lim_\limits{\lambda \to 0}\|f_{\...
Rajesh D's user avatar
  • 141
3 votes
0 answers
108 views

Are the No Free Lunch Theorems Useful for Anything?

I have been thinking about the No Free Lunch (NFL) theorems lately, and I have a question which probably every one who has ever thought of the NFL theorems has also had. I am asking this question here,...
Surgical Commander's user avatar
2 votes
1 answer
142 views

Machine Learning for Optimization

I have a function which takes 100+ coefficients and outputs $x$. I wish to optimise $x$. Running the simulation 50 000 times will take around 15 minutes, however, this happens in parallel - and the ...
Tomi's user avatar
  • 123
2 votes
2 answers
189 views

Handling inconsistent solutions obtained by PCA

In order to achieve a 2D representation $X\in\mathbb{R}^{n\times 2}$ of some high-dimensional data residing in $Y\in\mathbb{R}^{n\times k}$, I use PCA:$$X=Y\cdot U,$$where $U\in\mathbb{R}^{k\times 2}$ ...
usero's user avatar
  • 1,663
2 votes
2 answers
133 views

Using low rank property for maximal/minimal value search (or sorting)

I was thinking about the following problem: Suppose there is a positive semidefinite matrix $X$ of size $n$ (for example, a kernel). Suppose $X$ can be approximated as a low rank matrix, $X\approx ...
Gil's user avatar
  • 392
2 votes
2 answers
102 views

Neural Networks: what's the point of learning features that don't linearly separate?

Unless I'm mistaken, deep neural networks are good for learning functions that are nonlinear in the input. In such cases, the input set is linearly inseparable, so the optimisation problem that ...
user avatar
2 votes
2 answers
141 views

A sufficient number of distances to recover relative positions of n points

On several places I found different claims on a sufficient number of distances to recover relative positions of $n$ points in $d$-dimensional space. For instance, work from http://www.dimitris-...
usero's user avatar
  • 1,663
2 votes
1 answer
72 views

How to filter customer voice from customer - agent conversation recordings?

I have a doubt on the project I'm working now. Actually I want only customer voice from the recordings which contains customer-agent conversation.But I have no idea to filter customer voice from ...
sowmiya's user avatar
  • 21
2 votes
1 answer
177 views

Variational loss of hp-Variational Physics Informed Neural Networks for 2D-Poisson Equation in Tensorflow

I am trying to reproduce the results from the hp-VPINN paper (https://arxiv.org/pdf/2003.05385.pdf) on tensorflow (v1) for Poisson's equation, particularly the two-dimensional Poisson equation. In one ...
M.V.'s user avatar
  • 21
2 votes
1 answer
27 views

How To Interpret PCA Points Labeled With Specific Data Dimensions

I've done some PCA on my own, and am familiar with the basic concepts of how PCA components are calculated and applied. However, I'm working on a research project and am confused as to how to ...
Zachary Rohman's user avatar
2 votes
1 answer
137 views

Why can't we just use machine learning to select which model to use for a given model?

Suppose you have a classification problem, now what if I implement and train all classification models like logistic regression, KKN, naive Bayes, decision tree or random forest on the training data ...
Devesh Yadav's user avatar