10
They are not interchangeable.
Computational science tends to refer more to HPC, simulation techniques (differential equations, molecular dynamics, etc.), and is usually referred to as scientific computing.
Data science tends to refer to computationally-intensive data analysis, like "big data", bioinformatics, machine learning (optimization), Bayesian ...
8
I would suggest that a full database may be overkill for your purposes, though it would certainly work.
Even $5 \cdot 10^5$ rows should be no more than around 25mb of data.
I would strongly recommend doing the analysis/plotting/etc with the same tool that you will use for querying your data. It is my experience that when changing what to analyse only takes ...
8
I realize this question was asked a while ago, but I recently needed the Freschet distance as well.
I couldn't find any implementations for Python, so I wrote my own based on the paper: "Computing Discrete Frechet Distance" by "Thomas Eiter and Heikki Mannila", and thought I would share it for future reference.
It's written in Cython (save as frechet.pyx)
#...
5
I agree with Davidmh that calculating uncertainties should not be handled by an automatic library. You will very quickly run into a case where the automatics fail (try doing a Fourier transform for instance).
You say however that you just want to keep the uncertainties with your data. Why not just add them as an extra column in your dataframe? This is how I ...
5
Take a look at active subspaces, e.g., Active Subspace Methods in Theory and Practice: http://epubs.siam.org/doi/abs/10.1137/130916138
And a PDF here:
http://inside.mines.edu/~pconstan/docs/constantine-asm.pdf
I have a SIAM book (Active Subspaces: Emerging Ideas for Dimension Reduction in Parameter Studies) coming out in March.
Suppose $f$ maps $\mathbb{...
5
Let $\mathbf{\theta}$ be a Gaussian random vector with mean $\mathbf{\mu}$ and covariance matrix $\mathbf{\Sigma}_\mathbf{\theta}$. Let $\mathbf{p}_\theta$ denote the joint PDF.
Let $J_\mathbf{\theta}$ be the objective function, as its negative logarithm:
$$J_\mathbf{\theta}=-ln(\mathbf{p}_\theta)$$
By taking the partial derivatives w.r.t. $\theta_d$ and ...
5
When solving a nonlinear equation
$$ x = f(x), \qquad x\in\mathbb{R}^m, $$
the iteration method consists of generating successive guesses
$$ x_1 = f(x_0), \quad x_2 = f(x_1), \qquad x_k = f(x_{k-1}). $$
If this converges to a limit $x_*$, that limit will satisfy the equation $x_* = f(x_*)$. Convergence is guaranteed by Taylor's theorem so long as $|f'|<1$...
5
I highly recommend using a tool such as Sumatra for this. I used to have a similar "pedestrian" approach to yours for keeping track of many simulation runs with varying parameters, but in the end it just becomes a huge mess because it's next to impossible to design such an ad-hoc approach correctly upfront and to anticipate all the use cases and extensions ...
5
The discrete Fourier transform for a signal of period $T$ with $N$ samples reads in its inverse or reconstruction form as
$$
y(t)=\frac1{N}\sum_{k=-N/2}^{N/2}c_k e^{i2\pi k\frac{t}{T}}
$$
with redundancy in $c_N$ and $c_{-N}$ if $N$ is even. Sampling this at points $t_m=\frac{mT}{N}$ gives a completely determined linear system whose solution is given by the ...
4
No, it's illegal by the Fortran standard, but that said most compilers will let you get away with it (if the debugging options are not turned on and provided a correct interface is not in scope at the calling point) as long as you are passing Double Complex to double precision and the arrays of the later are twice the size of the former, and similarly ...
3
I would recommend looking at the griddata method in SciPy, which seems to have the functionality you need. Pay attention to the 'fill value' argument if you are looking at points outside your $x,y$ data set.
3
The main purpose of sampling tons of pseudo-random data as opposed to non-random data is related to Runge's phenomenon for polynomial interpolation: Uniform spacing of interpolation points is often a bad idea. But choosing better interpolation points require knowledge of the function you want to interpolate (or integrate etc.). If you don't have that ...
2
You may check my answer on Cross Validated. I didn't want to copy it here. Basically, you can use fast, randomized SVD to compute PCA basis and coefficients.
2
It seems that both $x$ and $y$ have an offset, since a Lissajous curve should oscillate around the origin, so: $x=A\sin{\left(at+\delta\right)}+x_0$ and $y=B\sin{\left(bt\right)}+y_0$.
Besides that, your test equations do not seem to be correct. Because the ratio $\frac{a}{b}$ determines the number of "lobes" (points where the curve crosses itself) of the ...
2
Managing uncertainties is actually a quite delicate statistics problem. The known expression for error propagation using squared partial derivatives is good when the errors are normally distributed, independent, and small. This is usually the case; and in fact, even if the normality or independence are not fully satisfied, for most practical cases the result ...
2
The cumulative distribution function is the integral (antiderivative) of the probability distribution function. In other words, the PDF is the derivative of the CDF. You can therefore compute the PDF by computing the derivative of your data, for example by forming a difference quotient to approximate the derivative from a finite set of points.
2
The input for the Gonzalez algorithm is a set of elements to cluster and the distance between every pair of them. The first step of the Gonzalez algorithm is to select an element at random, this will be your first center (lets call it c_1). Since you already have a center, you must compute the distance from every element to c_1. Next, select as a new center ...
2
Your reformulation is wrong: The product rule for derivatives is $(uv)' = u'v + uv'$. Applying this in your situation (with $\partial_x$ instead of $\frac{\partial}{\partial x}$ and $\partial_{xy} = \frac{\partial^2}{\partial x\partial y}$ for brevity would yield
$$\partial_{xy}(uv) = \partial_y(\partial_x u \,v + u\,\partial_x v)
= \partial_{xy}u \,v + \...
2
Bagging, Boosting, and Bayesian Model Averaging/Combination are all widely used techniques for doing this. These are discussed in many textbooks on machine learning.
2
If you understand the basics of Python, you should just try to get your hands dirty using some well known/documented Python libraries for the type of work you want to do.
If you find you don't understand how to use certain features of the libraries due to inadequate knowledge in Python, just study up on those features and then get back to working with the ...
2
This is a typical use case for a paired t-test. The idea is to consider only the runtime difference $\Delta t$ for each problem and test for the null hypothesis $E(\Delta t)=0$. For a step-by-step explanation, see e.g. (the article refers to segmentation evaluation, but on an abstract level the problem is identiclal to yours):
Mao, Kanungo: "Empirical ...
2
I am writing a general answer about porting a program running on a CPU to a GPU or FPGA.
Both GPU programs (using say CUDA) and CPU programs are written in high level languages like C, C++. Therefore it is much easier to port a CPU program to its equivalent on a GPGPU. The algorithm that you have presented seems suitable for porting to GPU. It is compute ...
2
Welcome to SE SciComp. First of all, I would suggest using Jupyter so that you have access to IPython and its nice timing magics (see %time and %timeit magic function). These magics take into account that you need to run the code a couple of times to get reliable measurements. If you need even more in-depth comparison, you also need to take into account ...
2
You should be able to store everything in one table and plot from that if your data is tidy. Check out this paper, it's only about 20 pages.
2
Here the solution, a function to create a randomized time-series starting from a PSD:
def gen_pkfield(nsample=100, alpha=-11.0 / 3, fknee=1, fsample=2.5): # sampling per block
ufreq = np.fft.fftfreq(n=nsample+1, d=1./fsample)[1:] # the Discrete Fourier Transform sample frequencies (with negative values), [1:] to solve the DC
kfreq = np.sqrt(...
1
I guess that you are not actually interested in the variance, but in a confidence interval for your observable $\theta$. It should be noted that computing the confidence interval from the variance (i.e. $\hat{\theta}\pm 2\sigma(\theta)$) is only guaranteed to work when you estimate $\theta$ with a Maximum-Likelihood estimator.
The problem with your "block" ...
1
Useful inclusion measures depend on whether you want to check pairwise similiarity or containment. For similiarity the Jaccard index is intuitive. For containment I would adapt it to what ratio of columns of my 'gold standard file' are in the file to check which is a common feature used in natural language problems.
For the order of columns in a file, an ...
1
The question on the minimal # of data points from the second code, the "forward" solver, is a very broad one.
It depends on the complexity of the underlying phenomenon.
In the simplest case, the flow is laminar and the pollutant particles are effectively "infinitesimal"; I guess you can approximate this with Fick diffusion (+ mass transport, so a ...
1
I doubt that such a built-in function exists in MS Excel. Nevertheless, this problem is a linear regression that is simple enough to solve analytically.
Let us start with
$$\Pi = \sum_{i=1}^{n} \left[y_i - \left(a x_i + b + \frac{c}{x_i}\right)\right]^2 \enspace $$
with $n$ the number of data pairs and $(a,b,c)\equiv (c_1, c_0, c_{-1})$. To find the ...
1
A practical complication of such travel time estimates is the time spent transferring from one line/train to another.
If all you aim for is the minimum travel time, then there are simple relatively fast algorithms to find one or all of the "optimal" routes. See Dijkstra's shortest path algorithm for a single start and destination, or Floyd-Warshall ...
Only top voted, non community-wiki answers of a minimum length are eligible