Single precision floating point numbers take up half the memory and on modern machines (even on GPUs it seems) operations can be done with them at almost twice the speed compared to double precision. Many FDTD codes that I have found exclusively use single precision arithmetic and storage. Is there a rule of thumb of when it is acceptable to use single precision for solving large-scale sparse systems of equations? I assume it must heavily depend on the matrix condition number.
Furthermore, is there any effective technique which uses double precision where necessary and single where the accuracy of double is not required. For instance, I would think that for a matrix vector multiplication or a vector dot product, it might be a good idea to accumulate the results in a double precision variable (to avoid cancellation error), but that individual entries to be multiplied with each other can be multiplied using single precision.
Do modern FPU's seamlessly allow conversion from single precision (float) to double precision (double) and vice versa? Or are these costly operations?