Where did you get that impression? I think your observation is more a side-effect of the classical problem domains for each class of methods. Classical applications of FEM, such as large-deformation elasticity, are indeed well-suited to Lagrangian formulations, while many classical applications of FVM, such as fluid dynamics, are most commonly treated in Eulerian reference frames. Even so, there have been many publications on Eulerian FEM methods for problems with moving boundaries (e.g., immersed boundary), as well as moving mesh methods with finite volume methods. Note that in general, ALE methods are expected to satisfy a geometric conservation law (GCL). The GCL is perhaps more natural to enforce in a FV setting, but FE is usually more forgiving of inexact satisfaction. Note that FV methods are commonly used in turbo-machinery, which usually has multiple moving meshes (e.g., one fixed and one rotating) with fluxes evaluated across interfaces between the moving meshes.
In general, methods that conform to interfaces allow use of fewer degrees of freedom to accurately represent the nonsmoothness at the interface than interface tracking methods. Thus, when it is practical to conform to an interface, that is often preferable. FE methods for solid mechanics are almost always written for unstructured meshes, and the interfaces usually do not become too deformed. We may simulate until some structure fails, but many analyses do not continue simulating the dynamic crumpling after failure. In the fluids world, interfaces more frequently change topology very irregularly, such as when simulating vigorously rising bubbles. Meshing all those interfaces and transfering solutions to new meshes would be a headache, so is usually avoided. Other interfaces, such as along the edge of a wing on an aircraft, are extremely anisotropic (aspect ratio up to $\sim 10^6$) and either static or with static topology (though certainly moving in applications such as aeroacoustics). CFD simulations usually conform to boundaries like the wing, but use interface capturing or cut-cell technology for interfaces like the rising bubbles.
