Timeline for Parallel I/O options, in particular parallel HDF5
Current License: CC BY-SA 3.0
4 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Dec 20, 2011 at 9:12 | comment | added | khinsen | Good suggestion! This will be some work because it means rewriting the code, but it's probably worth it. | |
| Dec 14, 2011 at 22:42 | comment | added | milancurcic | @khinsen What you could do to test your hypothesis is read the file with a small number of CPUs, say between 1 and 8, and scatter the data to the rest. Do the profiling, see how much time you spend on I/O and how much on scatter. Vary the number of CPU readers and see what gives you best performance. | |
| Dec 14, 2011 at 10:20 | comment | added | khinsen | It's input that creates my scalability issue. I suppose that all the incoming requests from many nodes overload the NFS server, but I have no idea how to verify this hypothesis. | |
| Dec 12, 2011 at 23:27 | history | answered | milancurcic | CC BY-SA 3.0 |