In general, the answers is no.
Modern CPU's excel at problems which have high arithmetic intensity and are implemented using floating point arithmetic.
A few figures will help explain the discrepancy.
On the Haswell chip the MULPS instructions performs 8 single precision floating point multiplications. The latency is 5 CPU cycles and the reciprocal throughput is 0.5. This means that your have to wait 5 cycles for first instruction to complete, but if you have a stream of independent operands, then you can eventually retire 2 instruction pr. cycle. This yields a peak rate of 16 single precision multiplications pr. cycle. In comparison the IMUL instruction multiplies a single pair of 32 bit integers. The latency is 3 cycles and the reciprocal throughput is 1. If you have a stream of independent operands, then the peak rate is 1 multiplication pr. cycle.
Achieving the peak flop rate it only possible for a few very specialized types of computations, such as doing inner products on vectors which fit in the L1 cache on a single core. Matrix-matrix multiply, the kernel for all blocked implementations of Gaussian elimination frequently achieves 70-80% of the theoretical peak on the Top500 computers.
Sparse arithmetic typically has a very low arithmetic intensity and is incompatible with modern computer architecture. Computers on the Top500 list execute the sparse benchmark (Conjugate Gradient) at about 2% of the peak flop rate. The best computer on the list does does less than 20%.
You can make some sparse algorithms run at a high fraction of the peak flop rate, but it requires extremely aggressive optimizations which are tailored to the specific task at hand. Attempting to boost the computational speed with integer computations will just lower the arithmetic intensity even further.
It is possible to increase the accuracy of, say, double precision floating point calculations by clever algorithms which return the result, as if it had been computed using a smaller unit round off error, and then rounded to double precision. If extreme accuracy is required, then this approach is far more likely to be successful.