Perhaps it's better to say that it depends on the nature (details) of the algorithm and linear algebra implementation.
More critically, while deep neural networks (DNN) have become widely popular, there are many other valuable techniques offered by machine learning. Thus, this question requires an answer that caters to both - ML in general and DNN in specifics.
Additionally, the paradigms of machine learning can be divided into practitioners and scholars. Practitioners are concerned with delivering task performance, whereas scholars are interested in seeking answers and giving advices via investigations and numerical analysis. Scholars may apply numerical techniques to machine learning algorithms in ways that practitioners won't; this is why double-precision may sometimes be involved in scholastic research.
Deep neural network, in practice, does not involve matrix inversion at all. The main algorithm involved is backpropagation. The gradient is computed after each forward pass; it is then scaled with the step size parameter and added back to the parameter matrices on each layer. Typically single precision is good for training, and inference can do with very low precisions. Typically the "adaptive mixed precision" technique is used in DNN. Typically, double precision does not improve DNN task accuracy at all when compared to single precision.
That said, singular value decomposition (SVD) is an option (choice) in the implementation of a technique called Low-Rank Adaptation (LoRA), the latter being well known for cost-effective fine-tuning of deep neural network large language models (closely related to the Transformer architecture). Rather than performing the steps in a high floating point precision, much of the engineering innovation went into modifying the SVD algorithm such that a reasonable approximation can be derived using the lowest possible floating point precision. In other words, whenever a speed-accuracy tradeoff happens, a lot of engineering effort is committed in order to enable a variety of middle ground approaches.
Outside of DNN, there are still many other machine learning algorithms that are not based on neural networks, and those might need to use a higher precision because they are based on different linear algebra techniques. The least squares method is probably the best well-known example.
Another relevant concern is numerical reproducibility. In DNN, numerical reproducibility is not highly valued. Instead, practitioners are fundamentally more concerned about replicating the task performance (accuracy) when a model is trained with a specific data set, a carefully described DNN architecture, and a detailed description of the training process. Training with stochastic gradient descent (SGD), minibatch, and data augmentation (data perturbations and alterations intended to make the model more task-robust) already involves tons of randomization; DNN practitioners are far more concerned about these stochastic techniques leading to a DNN model with good task performance (e.g. correctly classifying an object and its bounding box, without requiring subpixel accuracy) than the numerical reproducibility of the trained model parameters.
I've heard that KL divergence can be used to find an accurate approximation between a layer implemented with higher precision and a layer with lower precision, even when nonlinearity (e.g. multiple layers of DNN) is involved. This technique is often implemented into DNN frameworks to allow speed-accuracy tradeoffs. Unfortunately, I'm not familiar with these details.