Skip to main content
Computational Science

Return to Question

Tweeted twitter.com/#!/StackSciComp/status/322528424067465216
deleted 9 characters in body; edited title
Source Link
James Last
James Last
  • 131
  • 3

Doing How to do local FFT on huge 3D vector data cell mesh and visualizingvisualize it spatially?

The binary representations are IEEE floating point in network byte order (MSB). To insure that the byte order is correct, and to provide a partial check that the lele hasn't been sent through a non 8-bit clean channel, the firstfirst datum is a prede nedpredened value: 1234567.0 (Hex: 49 96 B4 38) for 4-byte mode, and 123456789012345.0 (Hex: 42 DC 12 21 83 77 DE 40) for 8-byte mode. The data immediately follow the check value. The structure of the data depends on whether the \meshtype" declared in the header is \irregular" or \rectangular". For irregular meshes, each data element is a 6-tuple, consisting of the x, y and z components of the node position, followed by the x, y and z components of the fieldfield at that position. Ordering among the nodes is not relevant. The number of nodes is speci edspecied in the \pointcount" line in the segment header. For rectangular meshes, data input is fi eldfield values only, in x, y, z component triples. These are ordered with the x index incremented firstfirst, then the y index, and the z index last. This is nominally Fortran order, and is adopted here because commonly x will be the longest dimension, and z the shortest, so this order is more memory-access ecientecient than the normal C array indexing of z, y, x. The size of each dimension is speci edspecied in the \xnodes, ynodes, znodes" lines in the segment header.

Doing local FFT on huge 3D vector data cell mesh and visualizing it spatially?

The binary representations are IEEE floating point in network byte order (MSB). To insure that the byte order is correct, and to provide a partial check that the le hasn't been sent through a non 8-bit clean channel, the first datum is a prede ned value: 1234567.0 (Hex: 49 96 B4 38) for 4-byte mode, and 123456789012345.0 (Hex: 42 DC 12 21 83 77 DE 40) for 8-byte mode. The data immediately follow the check value. The structure of the data depends on whether the \meshtype" declared in the header is \irregular" or \rectangular". For irregular meshes, each data element is a 6-tuple, consisting of the x, y and z components of the node position, followed by the x, y and z components of the field at that position. Ordering among the nodes is not relevant. The number of nodes is speci ed in the \pointcount" line in the segment header. For rectangular meshes, data input is fi eld values only, in x, y, z component triples. These are ordered with the x index incremented first, then the y index, and the z index last. This is nominally Fortran order, and is adopted here because commonly x will be the longest dimension, and z the shortest, so this order is more memory-access ecient than the normal C array indexing of z, y, x. The size of each dimension is speci ed in the \xnodes, ynodes, znodes" lines in the segment header.

How to do local FFT on huge 3D vector data cell mesh and visualize it spatially?

The binary representations are IEEE floating point in network byte order (MSB). To insure that the byte order is correct, and to provide a partial check that the le hasn't been sent through a non 8-bit clean channel, the first datum is a predened value: 1234567.0 (Hex: 49 96 B4 38) for 4-byte mode, and 123456789012345.0 (Hex: 42 DC 12 21 83 77 DE 40) for 8-byte mode. The data immediately follow the check value. The structure of the data depends on whether the \meshtype" declared in the header is \irregular" or \rectangular". For irregular meshes, each data element is a 6-tuple, consisting of the x, y and z components of the node position, followed by the x, y and z components of the field at that position. Ordering among the nodes is not relevant. The number of nodes is specied in the \pointcount" line in the segment header. For rectangular meshes, data input is field values only, in x, y, z component triples. These are ordered with the x index incremented first, then the y index, and the z index last. This is nominally Fortran order, and is adopted here because commonly x will be the longest dimension, and z the shortest, so this order is more memory-access ecient than the normal C array indexing of z, y, x. The size of each dimension is specied in the \xnodes, ynodes, znodes" lines in the segment header.

Source Link
James Last
James Last
  • 131
  • 3

Doing local FFT on huge 3D vector data cell mesh and visualizing it spatially?

Simulation type:

I'm running a simulation with the OOMMF micromagnetics package

http://math.nist.gov/oommf/

where are magnet is represented by a mesh of 3 million cells, it gets excited by a magnetic pulse, the single cells oscillate and data output for every cell is x, y, z direction and absolute magnetic field value of every component over time, so a mini-magnet. Now I want to do a FFT on m_z(t) for every cell and later visualize this magnet over time while cells with m_z(t) showing common oscillation modes/eigenfrequencies get same color.

Data output format:

it is "data binary 4" and as said x, y, z and absolute field value for every component is given for all time steps (below 300-500 time steops)

The manual says:

The binary representations are IEEE floating point in network byte order (MSB). To insure that the byte order is correct, and to provide a partial check that the le hasn't been sent through a non 8-bit clean channel, the first datum is a prede ned value: 1234567.0 (Hex: 49 96 B4 38) for 4-byte mode, and 123456789012345.0 (Hex: 42 DC 12 21 83 77 DE 40) for 8-byte mode. The data immediately follow the check value. The structure of the data depends on whether the \meshtype" declared in the header is \irregular" or \rectangular". For irregular meshes, each data element is a 6-tuple, consisting of the x, y and z components of the node position, followed by the x, y and z components of the field at that position. Ordering among the nodes is not relevant. The number of nodes is speci ed in the \pointcount" line in the segment header. For rectangular meshes, data input is fi eld values only, in x, y, z component triples. These are ordered with the x index incremented first, then the y index, and the z index last. This is nominally Fortran order, and is adopted here because commonly x will be the longest dimension, and z the shortest, so this order is more memory-access ecient than the normal C array indexing of z, y, x. The size of each dimension is speci ed in the \xnodes, ynodes, znodes" lines in the segment header.

I use a rectangluar mesh here

Question:

The FFT can be done using something like MATHEMATICA or MATLAB for a single cell and (m_z/ number, time/number) data list .dat file, but a data binary 4 file for a single time step for 3 million cells is up to 20 MB big, workstation with 30 GB RAM and 12 Cores available. The manual above mentions FORTRAN (to my knowledge not so easy to learn language although very fast).

What is the way to achieve the results I'm looking for and described above? From my point of view I face the problem to do the FFT in reasonable time with a script process for around 3 million cells and then transforming that data into a spectrogram (to see what different eigenfrequencies and higher harmonics exist, also this can be done by using the average m_z (summed over all single cells) and simply FFT it) but also have a output format of that data binary 4 format after FFT that can be used to feed a visualization software that can read such mesh data and color code it, show single 2D slices of the 3D object.

I read that MATHEMATICA can deal with binary 4 and offers high performance, but as this is completely new land to me, I first want to ask given above constraints and wishes, what is the best way and software/programming language to achieve this in reasonable time (few weeks)?