10
votes
Accepted
What is the difference between Computational Science and Data Science?
They are not interchangeable.
Computational science tends to refer more to HPC, simulation techniques (differential equations, molecular dynamics, etc.), and is usually referred to as scientific ...
- 11.6k
8
votes
Python implementation for Frechet Distance
I realize this question was asked a while ago, but I recently needed the Freschet distance as well.
I couldn't find any implementations for Python, so I wrote my own based on the paper: "Computing ...
- 285
8
votes
Accepted
Should I use a database to handle large amounts of results?
I would suggest that a full database may be overkill for your purposes, though it would certainly work.
Even $5 \cdot 10^5$ rows should be no more than around 25mb of data.
I would strongly recommend ...
- 2,473
5
votes
Accepted
Dimensionality reduction of the domain of f(x)
Take a look at active subspaces, e.g., Active Subspace Methods in Theory and Practice: http://epubs.siam.org/doi/abs/10.1137/130916138
And a PDF here:
http://inside.mines.edu/~pconstan/docs/...
5
votes
Should I use a database to handle large amounts of results?
I highly recommend using a tool such as Sumatra for this. I used to have a similar "pedestrian" approach to yours for keeping track of many simulation runs with varying parameters, but in the end it ...
- 151
5
votes
Accepted
Iterative Cunnigham correction in Millikans oil-drop experiment
When solving a nonlinear equation
$$ x = f(x), \qquad x\in\mathbb{R}^m, $$
the iteration method consists of generating successive guesses
$$ x_1 = f(x_0), \quad x_2 = f(x_1), \qquad x_k = f(x_{k-1}). $...
- 11.4k
5
votes
Why do problems arise in FFT for smaller value of df in Python?
The discrete Fourier transform for a signal of period $T$ with $N$ samples reads in its inverse or reconstruction form as
$$
y(t)=\frac1{N}\sum_{k=-N/2}^{N/2}c_k e^{i2\pi k\frac{t}{T}}
$$
with ...
- 3,601
4
votes
Is there a relationship between the covariance matrix and the partial derivative?
Let $\mathbf{\theta}$ be a Gaussian random vector with mean $\mathbf{\mu}$ and covariance matrix $\mathbf{\Sigma}_\mathbf{\theta}$. Let $\mathbf{p}_\theta$ denote the joint PDF.
Let $J_\mathbf{\...
- 2,139
4
votes
Accepted
FFT (Numerical Recipes in FORTRAN 77)
No, it's illegal by the Fortran standard, but that said most compilers will let you get away with it (if the debugging options are not turned on and provided a correct interface is not in scope at the ...
- 436
4
votes
Fitting gauss-hermite-parametrization to data?
You're not providing an initial guess for the parameters, and so optimize.curve_fit is defaulting to [1.,1.,1.,1.]. The solver ...
3
votes
Accepted
library for arithmetic operations on unstructured xyz
I would recommend looking at the griddata method in SciPy, which seems to have the functionality you need. Pay attention to the 'fill value' argument if you are looking at points outside your $x,y$ ...
- 714
3
votes
Accepted
What is "SOLVER" in R and Statistics/Analytics?
Generally, people refer to software packages as 'solver' that, well, solve some class of equations. For example, it could be stated that R contains a linear system solver.
- 176
3
votes
Accepted
Monte Carlo Simulation - Random Number Motivation
The main purpose of sampling tons of pseudo-random data as opposed to non-random data is related to Runge's phenomenon for polynomial interpolation: Uniform spacing of interpolation points is often a ...
- 11.9k
2
votes
Accepted
Finite difference for Lighthill source term
Your reformulation is wrong: The product rule for derivatives is $(uv)' = u'v + uv'$. Applying this in your situation (with $\partial_x$ instead of $\frac{\partial}{\partial x}$ and $\partial_{xy} = \...
- 11.9k
2
votes
Gonzalez algorithm
The input for the Gonzalez algorithm is a set of elements to cluster and the distance between every pair of them. The first step of the Gonzalez algorithm is to select an element at random, this will ...
- 21
2
votes
How are the outcomes that generated from different predictive models combined to get more accurate predictions?
Bagging, Boosting, and Bayesian Model Averaging/Combination are all widely used techniques for doing this. These are discussed in many textbooks on machine learning.
- 17.6k
2
votes
Accepted
Which statistical method should I use for comparing machine run-time of two algorithms?
This is a typical use case for a paired t-test. The idea is to consider only the runtime difference $\Delta t$ for each problem and test for the null hypothesis $E(\Delta t)=0$. For a step-by-step ...
- 481
2
votes
How much in depth should I cover Python?
If you understand the basics of Python, you should just try to get your hands dirty using some well known/documented Python libraries for the type of work you want to do.
If you find you don't ...
- 3,673
2
votes
GPGPU/FPGA programming for Combinatorial Analysis
I am writing a general answer about porting a program running on a CPU to a GPU or FPGA.
Both GPU programs (using say CUDA) and CPU programs are written in high level languages like C, C++. Therefore ...
- 121
2
votes
Is there a better way to do run time analysis than this?
Welcome to SE SciComp. First of all, I would suggest using Jupyter so that you have access to IPython and its nice timing magics (see %time and %timeit magic function). These magics take into account ...
- 6,066
2
votes
Accepted
Best way of storing numerical data in a compact manner, while leaving it accessible for tools like GnuPlot?
You should be able to store everything in one table and plot from that if your data is tidy. Check out this paper, it's only about 20 pages.
- 36
2
votes
Accepted
Bounds condition for IFT to obtain a $1/f$ time-series
Here the solution, a function to create a randomized time-series starting from a PSD:
...
- 61
1
vote
Converting ROOT Tree to HDF5
You might want to give a try to DataHarvester.
This tool claims to support ROOT, HBOOK, HDF, XML, SQLITE and input/output between them.
See the following publication:
W. Waltenberger, G. Richter, ...
- 8,362
1
vote
Python implementation for Frechet Distance
There is a discussion in Tyler Reddy's very recent pycon2017 tutorial on youtube https://youtu.be/ETJc3NfU9aA?t=9540
(at 2:39.00) where he discusses implementing the Frechet Distance in scipy.spatial. ...
- 11
1
vote
How are the outcomes that generated from different predictive models combined to get more accurate predictions?
Provided you have raw data you could use in this process, one could use the various different models and treat them as basis functions of sorts that you wish to merge together in a least square sense. ...
- 3,673
1
vote
Accepted
Is resampling more accurate than block average for statistical analysis of data?
I guess that you are not actually interested in the variance, but in a confidence interval for your observable $\theta$. It should be noted that computing the confidence interval from the variance (i....
- 481
1
vote
Similarity of two CSV files (or more?)
Useful inclusion measures depend on whether you want to check pairwise similiarity or containment. For similiarity the Jaccard index is intuitive. For containment I would adapt it to ...
- 156
1
vote
what is the best theory/model to use for prediction in multivariate data?
The question on the minimal # of data points from the second code, the "forward" solver, is a very broad one.
It depends on the complexity of the underlying phenomenon.
In the simplest case, the ...
- 121
1
vote
similarity/distance measurement between two ranked sequence
As a starter, I would use the cost of bringing the elements into the same order. Of course this ignores the insertion of missing elements.
A simple solution would be insertion sort:
...
- 111
1
vote
Accepted
Power series regression linear fit in VBA excel
I doubt that such a built-in function exists in MS Excel. Nevertheless, this problem is a linear regression that is simple enough to solve analytically.
Let us start with
$$\Pi = \sum_{i=1}^{n} \...
- 8,006
Only top scored, non community-wiki answers of a minimum length are eligible