I think you are missing a very important and crucial step that lies exactly between the physics and simulation: the mathematical model.
In order to model any physics, one has to formulate the mathematical description of the physical phenomenon. Depending on the goals of the simulation, different approximations and assumptions can be made resulting in various complexity of the systems and requiring more or less complicated numerical techniques to solve them.
You mentioned several very different areas of physics that you are interesting simulating, so, I guess, your interest lies more in the numerical methods/visualization rather than in any particular field. I will take electromagnetics (EM) as an example:
- the full glory of EM is covered by Maxwell's Equations
- various forms of which (and approximations) can be solved in 1D, 2D, and 3D.
- sometimes it is appropriate to use surface discretization (PEC - perfect electric conductor), sometimes you have to use volume discretization
- different properties of materials (nonlinearity, anisotropy, etc) might need to be taken into account.
What I am trying to say, is that to simulate any physics you first have to understand (to some extent) the mathematical model behind it.
Next, usually, you need to solve this model using some numerical technique. Depending on the model, different techniques can be applicable and preferable.
Again, I will use the ones that are commonly used for EM, but other fields will be similar:
- finite-difference (finite-difference time-domain)
- finite-element method (FEM)
- integral-equation methods (boundary element)
- physical/geometrical optics
...etc. Different libraries (including C++ ones) can be used to solve those problems: deal.ii, FEniCS...
However, my advice would be to start with something simpler and use a more appropriate language/platform to begin with numerical simulation of the physical phenomenon. Say, starting solving wave-equation in 1-D using Matlab (Octave) and visualizing simple propagation. Also using numerous already available toolboxes for PDE, ODE, and particular areas of physics should be easier there.
I would also suggest a book/course that has nothing to do with C++, but was my "bible" in computational science: