Questions tagged [machine-learning]

Machine learning, a branch of artificial intelligence, is the science of getting computer systems to meaningfully act without being explicitly programmed by human.

Filter by
Sorted by
Tagged with
16 votes
5 answers
5k views

Apply PCA on very large sparse matrix

I am doing a text classification task with R, and I obtain a document-term matrix with size 22490 by 120,000 (only 4 million non-zero entries, less than 1% entries). Now I want to reduce the ...
user avatar
16 votes
3 answers
4k views

Python OSS alternatives for Matlab Neural Network Toolbox. Any intercomparisons?

I'd like to be independent of commercial software for my scientific work. I find a dependence an commercial packages such as Matlab and its toolboxes unsatisfactory, because I do not know if I will ...
user avatar
  • 260
14 votes
3 answers
1k views

Why aren't Krylov subspace methods popular in the Machine Learning community compared to Gradient Descent?

Historically, iterative methods for solving relatively simple-structured systems $Ax=b$ with $A$ being a $4\times 4$ matrix or to find the eigenvalues of that matrix assuming in both problems that $A$ ...
user avatar
  • 297
12 votes
6 answers
4k views

Seeking a free symbolic regression software

Now that Formulize / Eureqa started charging $2500 a year for using it and having crippled the trial version, does anyone know of any replacements that can do similar things like find an equation ...
user avatar
  • 221
11 votes
1 answer
4k views

When training a neural network, why choose Adam over L-BGFS for the optimizer?

More specifically, when training a neural network, what reasons are there for choosing an optimizer from the family consisting of stochastic gradient descent (SGD) and its extensions (RMSProp, Adam, ...
user avatar
  • 113
10 votes
1 answer
1k views

Sensitivity of $y$ w.r.t. to $x$ in $y=f(x)$ where f is a routine

Given a model $f$ as a programming routine, such that we are able to compute $y=f(x)$ for any $x \in \mathcal{D}$, I am interested in the sensitivity (or let us say derivative) of $y$ with respect to $...
user avatar
  • 244
10 votes
2 answers
3k views

Do RBF kernel matrices tend to be ill-conditioned?

I use RBF kernel function to implement one kernel based machine learning algorithm(KLPP), the resulting kernel matrix $K$ $$K(i,j)= \exp\left({\frac{-(x_{i}-x_{j})^2}{ \sigma_{m}^2}}\right)$$ is ...
user avatar
  • 317
10 votes
2 answers
4k views

Calculating Lagrange coefficients for SVM in Python

I'm trying to write a full SVM implementation in Python and I have a few issues computing the Lagrange coefficients. First let me rephrase what I understand from the algorithm to make sure I'm on the ...
user avatar
9 votes
2 answers
184 views

Predict runtimes for dense linear algebra

I would like to predict runtimes for dense linear algebra operations on a specific architecture using a specific library. I would like to learn a model that approximates the function $F_{op} \;::\; $...
user avatar
  • 3,048
9 votes
2 answers
3k views

Markov (Chain) image generators?

Markov Chains can be used to generate, or auto-complete, text. https://en.wikipedia.org/wiki/Markov_chain#Markov_text_generators Training text is read, and some information about the text is ...
user avatar
  • 193
8 votes
2 answers
8k views

Use of machine learning in computational fluid dynamics

Background: I have only built one working numeric solution to 2d Navier-Stokes, for a course. It was a solution for lid-driven cavity flow. The course, however, discussed a number of schemas for ...
user avatar
8 votes
3 answers
5k views

fmincg implementation in Python

I'm trying to re-implement Neural Networks in Python. I implemented the cost function and the backpropagation algorithm correctly. I have checked them by executing its Octave equivalent code. But ...
user avatar
  • 223
7 votes
2 answers
229 views

Learning parameters of noise and filter coefficients from data where data and noise both have Gaussian distributions

Assume $X$ and $N$ are two sets of vectors (observations) from two different normal distributions, where $X$ represents clean data and $N$ represents noise; and $A$ a projection matrix of a filter. ...
user avatar
7 votes
4 answers
276 views

Testing for stability of a simulated dynamical system

Background and question I often work with simulations of dynamical systems and I usually track a single parameter $x$, such as the number of agents (for agents based models) or the error rate (for ...
user avatar
6 votes
1 answer
186 views

Support Vector Machines as Neural Nets?

This is more of a conceptual question. I have learned about Neural Nets, and I have some clue as to how Support Vector Machines work. I read somewhere however that given the appropriate kernel (is ...
user avatar
  • 165
6 votes
2 answers
159 views

Books/Resources on Sparse Optimization?

I'm looking to learn more about Sparse Optimization and apply it to machine learning problems. Could you please recommend some books/resources on this topic? Both theoretical and applied are fine.
user avatar
5 votes
2 answers
4k views

Are there tasks in machine learning which require double precision floating points?

Double-precision calculations are significantly slower or more expensive than single-precision calculations. For example, the NVidia Tesla which performs well on doubles is much more expensive then ...
user avatar
5 votes
4 answers
248 views

Learning computational science through guided discovery

I am currently trying to get through Pattern Classification by Duda et al (for a course). However, the book seems too dense for me. Pattern recognition seems like a topic that could be better learned ...
user avatar
  • 192
5 votes
1 answer
3k views

Logistic regression with Python

I am trying to code up logistic regression in Python using the SciPy fmin_bfgs function, but am running into some issues. I wrote functions for the logistic (...
user avatar
5 votes
1 answer
130 views

Using an approximation algorithm to adapt parameter values of a given algorithm

Problem: I have an incremental online clustering algorithm which need 4 parameters that should be specified by the user before execution. The algorithm will gives "good results" if "a good parameter ...
user avatar
5 votes
3 answers
262 views

Deep learning using Distributed linear algebra

Is there any deep learning library based on Trillinos or Petsc linear algebra?
user avatar
5 votes
1 answer
159 views

How to train a model to classify object trajectories?

I have a tracker that outputs the trajectory $(x,y,z)$ of an object (e.g., a can). I want to use these trajectories to train a classifier (i.e., SVM) in order to infer the activity that the person ...
user avatar
5 votes
1 answer
148 views

Looking for an understandable discussion of creating Maximum Entropy classifiers

Texts, articles, and papers on Maximum Entropy Classifiers tend to come in two varieties: the more popular "upper level", and the more technical. The popular variety are good at explaining the ...
user avatar
  • 658
4 votes
3 answers
294 views

Advantage of diagonal "jitter" for numerical stability?

In a machine learning code, that computes optimum parameters $\theta _{MLE}$ of a linear regression model, by maximum likelihood estimation: $$ \boldsymbol \theta^\text{ML} = (\boldsymbol\Phi^T\...
user avatar
  • 284
4 votes
1 answer
4k views

Using scipy.optimize to implement a neural network with back propagation

My problem is something similar to this. I'm trying to implement a (Neural Network) Cost function, Back propogation algorithm in Python. The Neural Network has 3 layers. Hence 2 parameters to optimize ...
user avatar
  • 223
4 votes
1 answer
160 views

Using the PAST algorithm to find eigenvectors

I'm working on trying to extract the eigenvectors from a series of observations from a random variable, by using the PAST algorithm, see e.g. 6.2.3 in this book: Large pdf. I don't understand the ...
user avatar
  • 41
4 votes
1 answer
1k views

Diffusion kernel "guide"

Diffusion kernels are kernels which "project" information about graphs into $R^n$ so that certain machine learning techniques can be performed. I have read through this paper and feel fairly ...
user avatar
  • 141
4 votes
0 answers
81 views

Identifying an unknown P.D.E. from solution data

I have a black-box simulation that produces the time evolution of a probability density function p(x, t) in 1 dimension from arbitrary initial conditions p(x, 0). The underlying simulation occurs on a ...
user avatar
  • 41
4 votes
0 answers
79 views

Calculus of Variations with unknown cost function but some data

I have a problem that I've framed out in a particular way, but I don't know if I'm re-inventing the wheel here. Is there an existing literature base in this problem? Does it have a corresponding term ...
user avatar
  • 141
3 votes
1 answer
2k views

Why am I not seeing faster neural network training after upgrading to a vastly better GPU?

I was previously running my neural networks using the Lasagne library to build and train neural networks in Theano on an NVIDIA GTX 750 Ti. I'm using a genetic algorithm to tune the hyperparameters of ...
user avatar
  • 141
3 votes
2 answers
1k views

Applying same feature selection to multiple data sets with Weka

I am using the Weka workbench to train a protein fold classifier. I imported my training data into Weka and performed PCA-based feature selection. This seems to have worked fine, but now I cannot ...
user avatar
3 votes
1 answer
94 views

Improve optimization over 'mapping' of indices

I have two tables at my disposal, one work dataset and one reference dataset. Each dataset has got two columns, lets say these are fields A and B. I would like to associate the rows in the reference ...
user avatar
  • 343
3 votes
2 answers
233 views

Optimization of expensive model with many parameters

I have a physical model which takes $\sim50$ parameters and gives $\sim2000$ outputs taking tens of minutes to run. I need to optimize these parameters to give outputs as close as possible to data. ...
user avatar
  • 131
3 votes
1 answer
766 views

Open source tool comparable to MATLAB Neural Network Tool Box?

Background I am a graduate student and a researcher. I do not like to rely on commercial products due to cost and also because I feel more comfortable in using and extending existing tools based on ...
user avatar
  • 131
3 votes
1 answer
1k views

How to train an L2-regularized L1 Hinge Loss SVM using vowpal wabbit?

I'm trying to train an L2-regularized L1-hinge loss SVM using vowpal-wabbit. I use the following commands to train and test on the splice dataset: ...
user avatar
3 votes
0 answers
116 views

A maximization problem, with motivation in machine learning

Consider the minimization problem described this paper. Let $f_{\lambda}$ be the minimizer. As a part of extending my work, I am able to show the following facts $$\lim_\limits{\lambda \to 0}\|f_{\...
user avatar
  • 141
3 votes
0 answers
105 views

Are the No Free Lunch Theorems Useful for Anything?

I have been thinking about the No Free Lunch (NFL) theorems lately, and I have a question which probably every one who has ever thought of the NFL theorems has also had. I am asking this question here,...
user avatar
2 votes
1 answer
128 views

Machine Learning for Optimization

I have a function which takes 100+ coefficients and outputs $x$. I wish to optimise $x$. Running the simulation 50 000 times will take around 15 minutes, however, this happens in parallel - and the ...
user avatar
  • 123
2 votes
2 answers
1k views

Interpolation vs. Neural network

I am seeking knowledge from the community. I am solving a transport PDE (conservation of solute mass) using COMSOL. At each Newton-Raphson iteration, I need to update a constant called $Kd$ for some ...
user avatar
  • 89
2 votes
2 answers
169 views

Handling inconsistent solutions obtained by PCA

In order to achieve a 2D representation $X\in\mathbb{R}^{n\times 2}$ of some high-dimensional data residing in $Y\in\mathbb{R}^{n\times k}$, I use PCA:$$X=Y\cdot U,$$where $U\in\mathbb{R}^{k\times 2}$ ...
user avatar
  • 1,633
2 votes
2 answers
123 views

Using low rank property for maximal/minimal value search (or sorting)

I was thinking about the following problem: Suppose there is a positive semidefinite matrix $X$ of size $n$ (for example, a kernel). Suppose $X$ can be approximated as a low rank matrix, $X\approx ...
user avatar
  • 372
2 votes
2 answers
92 views

Neural Networks: what's the point of learning features that don't linearly separate?

Unless I'm mistaken, deep neural networks are good for learning functions that are nonlinear in the input. In such cases, the input set is linearly inseparable, so the optimisation problem that ...
user avatar
2 votes
2 answers
136 views

A sufficient number of distances to recover relative positions of n points

On several places I found different claims on a sufficient number of distances to recover relative positions of $n$ points in $d$-dimensional space. For instance, work from http://www.dimitris-...
user avatar
  • 1,633
2 votes
1 answer
58 views

How to filter customer voice from customer - agent conversation recordings?

I have a doubt on the project I'm working now. Actually I want only customer voice from the recordings which contains customer-agent conversation.But I have no idea to filter customer voice from ...
user avatar
  • 21
2 votes
1 answer
23 views

How To Interpret PCA Points Labeled With Specific Data Dimensions

I've done some PCA on my own, and am familiar with the basic concepts of how PCA components are calculated and applied. However, I'm working on a research project and am confused as to how to ...
user avatar
2 votes
1 answer
131 views

Why can't we just use machine learning to select which model to use for a given model?

Suppose you have a classification problem, now what if I implement and train all classification models like logistic regression, KKN, naive Bayes, decision tree or random forest on the training data ...
user avatar
2 votes
1 answer
129 views

Correct Weights Neural Network

I'm trying to make a neural network using MATLAB. But I'm finding that many problems do not get solved. These are the formulas on which I based: $$ Layers\ in\ Neural\ Network = [0, ..., i, ..., out] ...
user avatar
2 votes
1 answer
116 views

Is there a way to return a substring of a string using Convolutional Neural Networks?

I'm a PhD student in genetics and molecular biology working on an algorithm to identify if a DNA sequence is either a transposable element (TE) or not a TE using convolutional neural networks, and it'...
user avatar
2 votes
1 answer
130 views

Artificial neural networks for Temperature prediction

Imagine I want to consider the temperature for a process given several input varibales. The temperature can be anywhere between 400 and 500 K. Consider I have experimental data to train the network ...
user avatar
  • 33
2 votes
1 answer
105 views

Using kalman filter when samples don't have time index

Assume $X$ and $N$ are two sets of observations from two different normal distribution, where $X$ represents clean data and $N$ represents noise; and $A$ a projection matrix of a filter and the ...
user avatar