Questions tagged [machine-learning]

Machine learning, a branch of artificial intelligence, is the science of getting computer systems to meaningfully act without being explicitly programmed by human.

Filter by
Sorted by
Tagged with
16
votes
5answers
5k views

Apply PCA on very large sparse matrix

I am doing a text classification task with R, and I obtain a document-term matrix with size 22490 by 120,000 (only 4 million non-zero entries, less than 1% entries). Now I want to reduce the ...
16
votes
3answers
4k views

Python OSS alternatives for Matlab Neural Network Toolbox. Any intercomparisons?

I'd like to be independent of commercial software for my scientific work. I find a dependence an commercial packages such as Matlab and its toolboxes unsatisfactory, because I do not know if I will ...
11
votes
6answers
4k views

Seeking a free symbolic regression software

Now that Formulize / Eureqa started charging $2500 a year for using it and having crippled the trial version, does anyone know of any replacements that can do similar things like find an equation ...
10
votes
1answer
1k views

Sensitivity of $y$ w.r.t. to $x$ in $y=f(x)$ where f is a routine

Given a model $f$ as a programming routine, such that we are able to compute $y=f(x)$ for any $x \in \mathcal{D}$, I am interested in the sensitivity (or let us say derivative) of $y$ with respect to $...
10
votes
2answers
3k views

Do RBF kernel matrices tend to be ill-conditioned?

I use RBF kernel function to implement one kernel based machine learning algorithm(KLPP), the resulting kernel matrix $K$ $$K(i,j)= \exp\left({\frac{-(x_{i}-x_{j})^2}{ \sigma_{m}^2}}\right)$$ is ...
10
votes
2answers
4k views

Calculating Lagrange coefficients for SVM in Python

I'm trying to write a full SVM implementation in Python and I have a few issues computing the Lagrange coefficients. First let me rephrase what I understand from the algorithm to make sure I'm on the ...
9
votes
2answers
180 views

Predict runtimes for dense linear algebra

I would like to predict runtimes for dense linear algebra operations on a specific architecture using a specific library. I would like to learn a model that approximates the function $F_{op} \;::\; $...
8
votes
2answers
8k views

Use of machine learning in computational fluid dynamics

Background: I have only built one working numeric solution to 2d Navier-Stokes, for a course. It was a solution for lid-driven cavity flow. The course, however, discussed a number of schemas for ...
8
votes
2answers
3k views

Markov (Chain) image generators?

Markov Chains can be used to generate, or auto-complete, text. https://en.wikipedia.org/wiki/Markov_chain#Markov_text_generators Training text is read, and some information about the text is ...
8
votes
3answers
5k views

fmincg implementation in Python

I'm trying to re-implement Neural Networks in Python. I implemented the cost function and the backpropagation algorithm correctly. I have checked them by executing its Octave equivalent code. But ...
8
votes
1answer
3k views

When training a neural network, why choose Adam over L-BGFS for the optimizer?

More specifically, when training a neural network, what reasons are there for choosing an optimizer from the family consisting of stochastic gradient descent (SGD) and its extensions (RMSProp, Adam, ...
7
votes
2answers
225 views

Learning parameters of noise and filter coefficients from data where data and noise both have Gaussian distributions

Assume $X$ and $N$ are two sets of vectors (observations) from two different normal distributions, where $X$ represents clean data and $N$ represents noise; and $A$ a projection matrix of a filter. ...
7
votes
4answers
274 views

Testing for stability of a simulated dynamical system

Background and question I often work with simulations of dynamical systems and I usually track a single parameter $x$, such as the number of agents (for agents based models) or the error rate (for ...
6
votes
1answer
182 views

Support Vector Machines as Neural Nets?

This is more of a conceptual question. I have learned about Neural Nets, and I have some clue as to how Support Vector Machines work. I read somewhere however that given the appropriate kernel (is ...
6
votes
2answers
153 views

Books/Resources on Sparse Optimization?

I'm looking to learn more about Sparse Optimization and apply it to machine learning problems. Could you please recommend some books/resources on this topic? Both theoretical and applied are fine.
5
votes
2answers
3k views

Are there tasks in machine learning which require double precision floating points?

Double-precision calculations are significantly slower or more expensive than single-precision calculations. For example, the NVidia Tesla which performs well on doubles is much more expensive then ...
5
votes
1answer
3k views

Logistic regression with Python

I am trying to code up logistic regression in Python using the SciPy fmin_bfgs function, but am running into some issues. I wrote functions for the logistic (...
5
votes
1answer
130 views

Using an approximation algorithm to adapt parameter values of a given algorithm

Problem: I have an incremental online clustering algorithm which need 4 parameters that should be specified by the user before execution. The algorithm will gives "good results" if "a good parameter ...
5
votes
1answer
154 views

How to train a model to classify object trajectories?

I have a tracker that outputs the trajectory $(x,y,z)$ of an object (e.g., a can). I want to use these trajectories to train a classifier (i.e., SVM) in order to infer the activity that the person ...
5
votes
1answer
148 views

Looking for an understandable discussion of creating Maximum Entropy classifiers

Texts, articles, and papers on Maximum Entropy Classifiers tend to come in two varieties: the more popular "upper level", and the more technical. The popular variety are good at explaining the ...
4
votes
3answers
244 views

Advantage of diagonal "jitter" for numerical stability?

In a machine learning code, that computes optimum parameters $\theta _{MLE}$ of a linear regression model, by maximum likelihood estimation: $$ \boldsymbol \theta^\text{ML} = (\boldsymbol\Phi^T\...
4
votes
4answers
237 views

Learning computational science through guided discovery

I am currently trying to get through Pattern Classification by Duda et al (for a course). However, the book seems too dense for me. Pattern recognition seems like a topic that could be better learned ...
4
votes
2answers
221 views

Deep learning using Distributed linear algebra

Is there any deep learning library based on Trillinos or Petsc linear algebra?
4
votes
1answer
4k views

Using scipy.optimize to implement a neural network with back propagation

My problem is something similar to this. I'm trying to implement a (Neural Network) Cost function, Back propogation algorithm in Python. The Neural Network has 3 layers. Hence 2 parameters to optimize ...
4
votes
1answer
160 views

Using the PAST algorithm to find eigenvectors

I'm working on trying to extract the eigenvectors from a series of observations from a random variable, by using the PAST algorithm, see e.g. 6.2.3 in this book: Large pdf. I don't understand the ...
4
votes
1answer
1k views

Diffusion kernel "guide"

Diffusion kernels are kernels which "project" information about graphs into $R^n$ so that certain machine learning techniques can be performed. I have read through this paper and feel fairly ...
4
votes
0answers
74 views

Identifying an unknown P.D.E. from solution data

I have a black-box simulation that produces the time evolution of a probability density function p(x, t) in 1 dimension from arbitrary initial conditions p(x, 0). The underlying simulation occurs on a ...
4
votes
0answers
79 views

Calculus of Variations with unknown cost function but some data

I have a problem that I've framed out in a particular way, but I don't know if I'm re-inventing the wheel here. Is there an existing literature base in this problem? Does it have a corresponding term ...
3
votes
1answer
2k views

Why am I not seeing faster neural network training after upgrading to a vastly better GPU?

I was previously running my neural networks using the Lasagne library to build and train neural networks in Theano on an NVIDIA GTX 750 Ti. I'm using a genetic algorithm to tune the hyperparameters of ...
3
votes
2answers
1k views

Applying same feature selection to multiple data sets with Weka

I am using the Weka workbench to train a protein fold classifier. I imported my training data into Weka and performed PCA-based feature selection. This seems to have worked fine, but now I cannot ...
3
votes
1answer
88 views

Improve optimization over 'mapping' of indices

I have two tables at my disposal, one work dataset and one reference dataset. Each dataset has got two columns, lets say these are fields A and B. I would like to associate the rows in the reference ...
3
votes
2answers
193 views

Optimization of expensive model with many parameters

I have a physical model which takes $\sim50$ parameters and gives $\sim2000$ outputs taking tens of minutes to run. I need to optimize these parameters to give outputs as close as possible to data. ...
3
votes
1answer
760 views

Open source tool comparable to MATLAB Neural Network Tool Box?

Background I am a graduate student and a researcher. I do not like to rely on commercial products due to cost and also because I feel more comfortable in using and extending existing tools based on ...
3
votes
1answer
1k views

How to train an L2-regularized L1 Hinge Loss SVM using vowpal wabbit?

I'm trying to train an L2-regularized L1-hinge loss SVM using vowpal-wabbit. I use the following commands to train and test on the splice dataset: ...
3
votes
0answers
112 views

A maximization problem, with motivation in machine learning

Consider the minimization problem described this paper. Let $f_{\lambda}$ be the minimizer. As a part of extending my work, I am able to show the following facts $$\lim_\limits{\lambda \to 0}\|f_{\...
3
votes
0answers
104 views

Are the No Free Lunch Theorems Useful for Anything?

I have been thinking about the No Free Lunch (NFL) theorems lately, and I have a question which probably every one who has ever thought of the NFL theorems has also had. I am asking this question here,...
2
votes
1answer
120 views

Machine Learning for Optimization

I have a function which takes 100+ coefficients and outputs $x$. I wish to optimise $x$. Running the simulation 50 000 times will take around 15 minutes, however, this happens in parallel - and the ...
2
votes
2answers
992 views

Interpolation vs. Neural network

I am seeking knowledge from the community. I am solving a transport PDE (conservation of solute mass) using COMSOL. At each Newton-Raphson iteration, I need to update a constant called $Kd$ for some ...
2
votes
2answers
157 views

Handling inconsistent solutions obtained by PCA

In order to achieve a 2D representation $X\in\mathbb{R}^{n\times 2}$ of some high-dimensional data residing in $Y\in\mathbb{R}^{n\times k}$, I use PCA:$$X=Y\cdot U,$$where $U\in\mathbb{R}^{k\times 2}$ ...
2
votes
2answers
108 views

Using low rank property for maximal/minimal value search (or sorting)

I was thinking about the following problem: Suppose there is a positive semidefinite matrix $X$ of size $n$ (for example, a kernel). Suppose $X$ can be approximated as a low rank matrix, $X\approx ...
2
votes
2answers
91 views

Neural Networks: what's the point of learning features that don't linearly separate?

Unless I'm mistaken, deep neural networks are good for learning functions that are nonlinear in the input. In such cases, the input set is linearly inseparable, so the optimisation problem that ...
2
votes
2answers
133 views

A sufficient number of distances to recover relative positions of n points

On several places I found different claims on a sufficient number of distances to recover relative positions of $n$ points in $d$-dimensional space. For instance, work from http://www.dimitris-...
2
votes
1answer
53 views

How to filter customer voice from customer - agent conversation recordings?

I have a doubt on the project I'm working now. Actually I want only customer voice from the recordings which contains customer-agent conversation.But I have no idea to filter customer voice from ...
2
votes
1answer
129 views

Why can't we just use machine learning to select which model to use for a given model?

Suppose you have a classification problem, now what if I implement and train all classification models like logistic regression, KKN, naive Bayes, decision tree or random forest on the training data ...
2
votes
1answer
128 views

Correct Weights Neural Network

I'm trying to make a neural network using MATLAB. But I'm finding that many problems do not get solved. These are the formulas on which I based: $$ Layers\ in\ Neural\ Network = [0, ..., i, ..., out] ...
2
votes
1answer
97 views

Is there a way to return a substring of a string using Convolutional Neural Networks?

I'm a PhD student in genetics and molecular biology working on an algorithm to identify if a DNA sequence is either a transposable element (TE) or not a TE using convolutional neural networks, and it'...
2
votes
1answer
21 views

How To Interpret PCA Points Labeled With Specific Data Dimensions

I've done some PCA on my own, and am familiar with the basic concepts of how PCA components are calculated and applied. However, I'm working on a research project and am confused as to how to ...
2
votes
1answer
128 views

Artificial neural networks for Temperature prediction

Imagine I want to consider the temperature for a process given several input varibales. The temperature can be anywhere between 400 and 500 K. Consider I have experimental data to train the network ...
2
votes
1answer
105 views

Using kalman filter when samples don't have time index

Assume $X$ and $N$ are two sets of observations from two different normal distribution, where $X$ represents clean data and $N$ represents noise; and $A$ a projection matrix of a filter and the ...
2
votes
1answer
84 views

Optimality criterion of PCA via recovered distances

It is stated in http://users.eecs.northwestern.edu/~yingwu/teaching/EECS510/Reading/Williams_NIPS01.pdf that the PCA mapping from $h$-dimensional data to low $k$-dimensional space minimizes $$\sum_{...