The short answer is: Yes.
Now for the long answer.
As the other answers point out, it is definitely possible but you would have to adjust your time step accordingly, which will make your simulation be extremely slow compared to if you were using an incompressible solver.
A few years back, I was doing an internship for a private research center which had develop a very robust compressible problem. I was using it to solve an incompressible solver. In order to do it as efficiently as possible (in terms of CPU time) as possible, I had to increase the velocity up to the incompressible limit (Mach $\approx 0.2$) and also increase the viscosity accordingly so the Reynolds number would remain unchanged: $ Re = \dfrac{v D}{\nu} $