# Questions tagged [data-analysis]

Data Analysis is the process of evaluating data using analytical and logical reasoning to examine each component of the data provided.

72 questions
Filter by
Sorted by
Tagged with
1 vote
157 views

### Storing Raw Simulation Data or Truncated Data?

I have a simulation that can generate quite a bit of data when it runs, for example $650\cdot 400 \cdot 400$ floating point numbers. Without compression, that's a few gigabytes worth if I want to save ...
36 views

### Dimensionality reduction between discrete wavelet families

I have what it may be a ridiculous question (since I don't know much about wavelets), but here I go. I am using different Discrete Wavelet families to extract texture features from images. I plan to ...
1 vote
84 views

### What are the Exact Rules for Significant Figures, Precision, and Uncertainty?

In the physical sciences (which are physics, chemistry, astronomy, materials science, etc.), we learned that the uncertainty is +/- the smallest unit (which is 1) of the last significant figure if the ...
123 views

### Fitting gauss-hermite-parametrization to data?

I want to fit this data. I have the following model functions. Classic gaussian: def gauss_model(x, mu, sigma): return np.exp(-0.5*((x-mu)/sigma)**2) And ...
46 views

### Bounds condition for IFT to obtain a $1/f$ time-series

I am coding a function to obtain a randomized time-series from a given $\frac{1}{f}$ law. The randomization is obtained by introducing a random phase in the function. I experience a problem in the ...
281 views

...
34 views

### Parameter explained by many distributions

If we had, for example, labeled data, where for each entry (label) we have several data distributions associated to it, how can I get something meaningful from them? Is this a solvable problem? Is ...
23 views

### I need help developing a process to map many individual records (from a database) into larger groups

There are many rows of data in a database table, each of which contains many columns. ID (numeric - 12345, 12378, etc) Name 1 - AAA,BBB,CCC Name 2 - dd, ee, ff Name 3 - Ggg, Hhh, Iii The objective ...
46 views

101 views

### How much in depth should I cover Python? [closed]

I'm an undergrad who's studying electronics and fields like data sciences and mostly computing seem most interesting to me but have in-depth knowledge in neither. I learnt basic python last winter ...
1 vote
129 views

### Implementing Frechet Distance on pandas data frame [duplicate]

I am working on a trajectory analysis project using python and its data science related libraries. I am planning to implement Frechet Distance algorithm for trajectory analysis, each trajectory has ...
1 vote
4k views

### Python implementation for Frechet Distance

I am working on a trajectory analysis project using python and its data science related libraries. I am planning to implement Frechet Distance algorithm for trajectory analysis, each trajectory has ...
1 vote
31 views

### Use custom defined metric(not matrix) in Javaplex

I'm about to get started trying to use Javaplex for the first time for some application in topological data analysis (TDA) and I want to use a custom defined metric. From looking over the documents, I ...
7k views

### What is the difference between Computational Science and Data Science? [closed]

Background: My PhD was in 'Computational Science'. My dissertation was on the analysis of X-Ray Diffraction Data and analysis of thermally perturbed nuclei in the overall dynamic analysis of the ...
43 views

### How are the outcomes that generated from different predictive models combined to get more accurate predictions?

The simple average is commonly used to combine the predictions out of different predictive models. Apart form the simple average, what are the other methods that can be used for combining the ...
1 vote
618 views

### FFT (Numerical Recipes in FORTRAN 77)

I am trying to use the subroutine twofft which in turn uses subroutine four1 in the book(Numerical Recipes in Fortran) to ...
496 views

### Is resampling more accurate than block average for statistical analysis of data?

I'm working in laboratories where molecular dynamics data are almost always analysed usign block average as stated in the famous Allen and Tildesley book. We divide the datas in blocks of size $M$ on ...
1k views

### Similarity of two CSV files (or more?)

A CSV file is characterized by a header, describing the n datarows to come. The header is a text string separated by a separator. A CSV header might look like (C1) ...
1 vote
71 views

### library for arithmetic operations on unstructured xyz

I've got several sets of xyz-data where x,y are coordinates and z is a 'value'. I want to be able to do simple arithmetic operations on these data. The x,y locations are not necessarily the same in ...
424 views

### what is the best theory/model to use for prediction in multivariate data?

I use software for pollutant propagation on rivers that takes as input a set of parameters ($p_1,p_2,\ldots,p_n$) and creates an output file which is basically a matrix where on each row the ...
1 vote
83 views

### Finite difference for Lighthill source term

I have experimental data containing horizontal and vertical components of speed and I need to evaluate this: $$\frac{\partial^2}{\partial x_i \partial x_j}\left(v_iv_j\right)$$ Denoting $U$ as ...
3k views

### Is there a relationship between the covariance matrix and the partial derivative?

Suppose that there are $N$ pieces of data, each of which contains $M$ parameter values such that $N >> M$. If we put this information into matrix form ($N$ rows, $M$ columns) and then compute ...
262 views

### similarity/distance measurement between two ranked sequence

Is there an efficient way to measure similarity/distance between two sequences of ranked numbers/letters. The two sequences are of different length, and only have some elements in common? For example,...
1 vote
45 views

### Manipulating/Extracting Data and Developing Methods - Language Choice [closed]

As a general programming enthusiast and aspiring Bioinformatician student I have an intermediate understanding of computing (languages) as well as Java, and to a lesser extent C++. Having knowledge in ...
342 views

### Iterative Cunnigham correction in Millikans oil-drop experiment

I tried to write an analysis program in python for the Millikan experiment based on a paper from Jones "The Millikan oil‐drop experiment: Making it worthwhile" However, I cannot reproduce the ...
666 views

### Power series regression linear fit in VBA excel

I wrote a program that calculates the best fit in VBA excel for the following model $$y_k=c_1x_k+c_0+c_{-1}(x_k)^{-1}$$ solving for the best fit parameters $c_1$, $c_0$, and $c_{-1}$. However I ...
47 views

### Simple Path to Route Algorithm

I'm currently conducting research into subway paths, but stand in front of a problem, with no programming knowledge and an extremely short time-limit. Probably the question I am asking is trivial, ...
159 views

### Can Box-Cox transformation be applied for data of this form?

I have data of the form: X Y 3.53 0 4.93 50 5.53 60 6.21 70 7.37 80 9.98 90 16.56 100 And I want to find out $n$ so that this can be ...
3k views

### Gonzalez algorithm

I'm getting confused about implementing the Gonzalez algorithm for k-center clustering. The algorithm is : let $S$ be a data set Choose $z_1 \in S$ arbitrarily. Number $z$ is center. (In general ...
939 views

### What is "SOLVER" in R and Statistics/Analytics?

I tried to research what exactly is a SOLVER only to find a not clear-cut simple answer. My doubts still remain after going through several sites full of discussions about it. I need to clarify as to ...
338 views

### Dimensionality reduction of the domain of f(x)

I'm wondering if there is something analogous to a PCA for data sets where there is a dependent variable. (Though I am interested in any method of dimensionality reduction, PCA is just an example.) ...
171 views

### Monte Carlo Simulation - Random Number Motivation

For Monte Carlo simulations, or any other numerical methods that rely heavily on the quality of the pseudo-random numbers generated (i.e even/desired distribution on a certain domain) for that matter, ...
721 views

### Should I use a database to handle large amounts of results?

Background: I am currently running a large amount parameter variation experiments. They are being run in Python 2.6+, using numpy. These experiments are going to take about 2 weeks to run. Roughly I ...
1 vote