I want to access a GPU on a computer remotely, and share this GPU with different desktops. I envision this sort of access as being similar to using a VNC. Is it possible to do something like what I've just described, and if so, how would I go about doing it?
VirtualGL/TurboVNC may help https://sourceforge.net/projects/virtualgl/. I haven't tried 3 VNC sessions at once but I think it should work. There are cloud solutions coming but are proprietory and you will have pay for them, it requires the hardware to handle sharing.
http://www.gputechconf.com/gtcnew/on-demand-gtc.php has some presentations on the tech.
Another thing that might help is rCUDA http://www.rcuda.net/