The tag has no wiki summary.

learn more… | top users | synonyms (1)

4
votes
7answers
175 views

Recommendations for input file format for science applications

Assuming that a program is written in either C, C++, Fortran or python, what do you think is the best format for the input file given that users could be relatively diverse, i.e. not familiar with the ...
3
votes
2answers
81 views

Cloud solutions for microscopy image analysis

At our research facility we routinely perform what are called PALM (Photo-activated localisation microscopy) or STORM (stochastic optical reconstruction microscopy) experiments. You can read all ...
5
votes
2answers
402 views

Pandas limitations and its alternatives in Python

I read somewhere that Pandas was first developed for the financial world, at least not especially for natural sciences (physics, biology and so on), so is there any similar data analysis Python ...
1
vote
1answer
46 views

Searching in a large dataset

I am working on a model of social interactions in mice. I have mice and boxes and a simulation that outputs which mouse stays in which box during which time period (this comes out as a mouse is ...
9
votes
2answers
185 views

suggestion for managing simulation runs?

This questions may be a bit off-topic in comp-sci. if it is needed please suggest where does it fit with. The question is regarding on how to manage all the simulation runs efficiently. let's say, ...
9
votes
3answers
366 views

Are HDF5 files suitable for git revision control?

I am not familiar with the file format used in HDF5, but I am wondering if HDF5 files are suitable for revision control with git (or for example Mercurial or Subversion)? I guess what I mean is: are ...
7
votes
2answers
732 views

Is there a fast way to compute histograms for high-dimensional large datasets?

Currently the way I compute histograms for data is by generating grid in $N$ dimensions (where $N$ is the dimension of the data) and searching through the $M$ data points in each dimension to see in ...
0
votes
0answers
946 views

Tecplot: Overlapping of 2 contour plot data from a single file, in a 2D Cartesian grid

I have a rectangular grid ASCII .dat file that looks like this ...
0
votes
1answer
47 views

mapping data with a spike to a heat map

I have the following dataset that I need to display on the heat map: [30, 15, 66, 7, 9999, 78, 42, 132] So if I map the values to the color scale using a linear function I only see the spike while ...
0
votes
3answers
155 views

Translating from SBML to Petri Net (matrix representation)

Is there a piece of software that can easily take a representation of a biochemical network from a SBML file and translate it into a Petri Net? Specifically, I'm looking for something that can give ...
0
votes
3answers
3k views

Dealing with non-monotonically increasing data

How do you deal with data that you need to be monotonically increasing in order to work with interpolation libs and other functions, when it is in fact not monotonically increasing?
4
votes
3answers
1k views

Storing a large, sparse array for R and Python

I've been working in R but sometimes switching to python. I'd like a more inter-language portable way of storing a large array than a csv file. (The particular csv file I'm dealing with is about 10^6 ...
9
votes
2answers
419 views

Best practice for storing hierarchical simulation data

TL,DR What is the accepted best practice in scientific computing circles for storing large quantities of hierarchically structured data? For example, SQL does not play nicely with large sparse ...
4
votes
2answers
225 views

Memory usage within graphics cards

I have a question about the memory usage within the graphics card. What does happen when the used application needs more memory than the graphic card does contain? let`s say the reserved main memory ...
5
votes
1answer
63 views

Are there any open standards for data exchange between components of a spatial model?

I'm just getting into climate research, and am amazed at how much poorly documented legacy code is used. This kind of thing means that models more or less have to be maintained by the people who wrote ...
28
votes
14answers
5k views

Which software and workflow is recommend for publication of scientific data and graphs?

Which software provides a good workflow from simple plotting of a few datapoints up to the creation of publication level graphics with detailed styles, mathematical typesetting and "professional ...
6
votes
2answers
295 views

Filtering a dataset to get a more uniform distribution for neural network training

I'm looking into using artificial neural networks (ANN) to predict the reaction rates in my fluid instead of solving the full system of stiff ODEs. Some people from my lab have already done some work ...
4
votes
3answers
223 views

How to fill a 2D set over a cartesian lattice with as few rectangles as possible?

Suppose I have a black and white image (composed of binary pixel values in a 2D cartesian array) that contains an irregular, nonconvex shape. Let's further suppose that the shape is one connected ...
1
vote
3answers
489 views

List of data munging libraries

I am trying to put together a complete list of all of the tools that computational scientists have found useful when trying to munge data, i.e. take data in one format, extract the useful bits, and ...
13
votes
4answers
288 views

How to deal with too much data?

Our plasma dynamics simulations often produce too much information. During the simulations we record various physical properties on a grid (x,y,z,t) that is as large as (8192x1024x1024x1500), for at ...