Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [linear-algebra]

Questions on the algorithmic/computational aspects of linear algebra, including the solution of linear systems, least squares problems, eigenproblems, and other such matters.

0
votes
1answer
31 views

Finding Matrix inverse with LU and repeted left division calls

Hello I am in a basic numerical methods class and our teacher has given us an algorithm which can compute the inverse of a matrix other than using MATLAB's built in library function. ...
6
votes
1answer
98 views

numerical solution of an under-determined linear equation in high dimensions

I need to solve a linear regression problem $$Ax=y$$ which is hugely underdetermined. I have around $10^6$ features but only $10^3$ equations. So $A$ is a $1,000\times 1,000,000$ matrix and $y$ a ...
2
votes
1answer
91 views

Projection onto the set of Orthogonal matrices

Let $M \in \mathbb{R}^{n \times n}$ and denote the set of Orthogonal matrices by \begin{equation} \mathcal{O}_{n} = \left\lbrace Q \in \mathbb{R}^{n \times n} \colon QQ^{T} = \mathbb{I}_{n} \right\...
2
votes
2answers
86 views

Asymptotic Complexity of Gaussian Elimination using Complete Pivoting

I would like to know the algorithm asymptotic complexity with Complete Pivoting. With partial pivoting, it is known to be $O(n^3)$. Is it the same for complete pivoting?
1
vote
2answers
102 views

Condition number of a matrix

I saw some definition that if $$M=\max_{x\not=0} \tfrac{\|Ax\|}{\|x\|} \quad\text{and}\quad m=\min_{x\not=0} \tfrac{\|Ax\|}{\|x\|} \,,$$ the condition number of $A=M/m$. In my opinion, If we have a ...
6
votes
1answer
66 views

Solving a Hadamard + Linear System

I need to solve a matrix equation which contains a Hadamard product and a standard matrix multiplication: $$A\odot X + BX = C$$ where $A, C, X \in \mathbb{R}^{m \times n}$, $B \in \mathbb{R}^{m \...
9
votes
3answers
148 views

What is the reason that LAPACK uses $\tau$ in QR decomposition (instead of normalizing the reflection vector)?

LAPACK's QR routine stores Q as Householder reflectors. It scales the reflection vector $v$ with $1/v_1$, so the first element of the result becomes $1$, so it doesn't have to be stored. And it stores ...
2
votes
1answer
131 views

Which C++ linear algebra library is probably the fastest on solving huge sparse [square matrix] linear system?

I am developing a 2D CFD solver for fluid-particle interaction. To solve Navier-Stokes equations on a grid of size $10000\times 10000$ cells (or >1 million cells), a large linear system $Ax=b$ with $A$...
4
votes
1answer
82 views

Numerically find the nearest positive semi definite matrix to a symmetric matrix

I have a symmetric matrix $M$ which I want to numerically project onto the positive semi definite cone. To do so, I decompose it into $M = QDQ^T$ and transform all negative eigenvalues to zero. (...
5
votes
1answer
88 views

Nullspace calculation of large matrix with rational numbers without round-off errors (exact)

I need to calculate a basis of the nullspace of large (up to a thousand columns and rows) matrices. For my application, it is very important that no round-off errors occur during the computation, so I ...
1
vote
1answer
84 views

Computing the Inverse of a matrix, using the Cholesky decomposition

I have to compute $CA^{-1}B$ and $CA^{-1}x$, where $A,B,C$ are conformable matrices and $x$ is a vector. I've read that the a very computationally stable way to compute these inverses is by computing ...
3
votes
0answers
58 views

Block matrix and DSYRK

I want to compute the matrix $$ A = \sum_{i=1}^N v_i v_i^T $$ where each $v_i$ is a given vector of length $2500$, so that $A$ is $2500 \times 2500$, and my $N$ is about 2 million. Rather than call ...
2
votes
2answers
153 views

Best software to do big number calculations quickly

I am trying to do some work on some math conjecture. I am testing the conjecture numbers using very large math numbers (100+ digits ). I am currently using python to test these numbers. In the ...
2
votes
0answers
48 views

Discrete-time Algebraic Riccati Equation (DARE) solver in C++

I need to use a Discrete-time Algebraic Riccati Equation (DARE) solver for an embedded controller (with limited processing power) in a research project and sadly, I can't find any implementation of it ...
2
votes
1answer
78 views

Checking positive definiteness on a hyperplane

Is there a faster way to check whether $A\in\mathbb{R}^{n\times n}$ is positive definite on $b^{\bot}:=\{x\in \mathbb{R}^{n}: x\cdot b=0\}$ than ...
1
vote
1answer
93 views

Operation count for GMRES

One can use GMRES as it is, but there is also a version of GMRES called k-step restarted GMRES, which is used for large matrices, where $k$ is some fixed number of steps after which we take a new $x_0$...
3
votes
1answer
91 views

How to find the nearest/a near positive definite from a given matrix?

I'm given a matrix. How do I find the nearest (or a near) positive definite from it? The matrix can have complex eigenvalues, not be symmetric, etc. However, all its entries are real valued. The ...
4
votes
0answers
60 views

Improving convergence of Jacobi iteration to Schur form

I'm using SIMD processor arrays to compute the eigen-decomposition for large numbers of small (up to $32\times 32$) matrices. For assorted technical reasons, Jacobi iteration maps well to the SIMD ...
3
votes
1answer
71 views

Condition number of matrix and effects of round off errors

In my numerical linear algebra class, I learned that for some matrices, it could have an element that is a very small number that is approximately 0 (and many orders of magnitude different from all ...
2
votes
0answers
80 views

A Bound for the inverse of the sum of identity and triangular matrix

I wonder if there are any theorems which can help me to calculate an upper bound for the spectral norm of: $$\left\| \left[ I + \sum_{i=1}^{\overline{n}\in\mathbb{N}} \big( C_i - I\big)\right]^{-1}\...
4
votes
1answer
115 views

Fast matrix multiplication with matrix elements computed on-the-fly (without forming the matrix)

Is there any library or routine for high-performance matrix-matrix product, where the matrix elements are computed on-the-fly using a given function of $i$ and $j$? More specifically, in the problem ...
3
votes
0answers
79 views

Why the MIRACLE of Lanczos/CG-like?

Lanczos/Arnoldi/Rietz/CG-like algorithm share the same core strategy... In each, a little miracle appears, most of the Gram-Schmidt inner products are zeroes! In others words, new direction need only ...
1
vote
1answer
82 views

Multigrid preconditioner for conjugate gradient methods

When solving $A*x=b$ using preconditioned conjugate gradient methods one has to solve $z=K^{-1}*r$ for the preconditioning where $K$ is the preconditioner of $A$ and $r$ is the residual vector. ...
3
votes
0answers
64 views

Element Preconditioner

Im just working on a preconditioner for the linear equation system $Ax = b$ arising in FEM for elliptic PDE. $A$ is a s.p.d Matrix with real valued entries. I read something about the element by ...
-1
votes
1answer
67 views

Relation of Condition of a Matrix and Convergency

Can anybody explain me the relation between the condition of a Matrix and the convergency of a problem. For example how is the relation between the condition of the stiffness Matrix occuring in FEM ...
3
votes
2answers
118 views

Factoring the sum of two matrices

Given \begin{equation} A_i=B+C_i \end{equation} where $A_i$,$B$ and $C_i$, $i=1,\dotsc,N$ are large square matrices, $B$ is symmetric, $C_i$ are zero matrices aside for a square block on the diagonal. ...
0
votes
1answer
42 views

Slightly change two vectors to satisfy a constraint

$\vec{a}\cdot\vec{b} \approx c$ $\vec{\alpha} \cdot \vec{\beta} = c$ $\vec{\alpha}$ is close to $\vec{a}$ and $\vec{\beta}$ is close to $\vec{b}$ Given $\vec{a}$, $\vec{b}$ and c, how to find $\vec{\...
7
votes
2answers
158 views

How does the QR algorithm applied to a real matrix returns complex eigenvalues?

I'm a noob into eigenvalues algorithms, but something call my attention. QR algorithm works with real/complex matrices producing real/complex eigenvalues. However, it can not produce complex ...
1
vote
1answer
72 views

Why does Newton's method with Linear Equality Constraints use KKT condition?

Goal: Optimize convex function $f(\vec{x})$ subjected to constraint $A\vec{x} = \vec{b}$ starting at a point $\vec{x}_0$ that satisfies the constraint. The problem only has equality constraint. Why ...
2
votes
0answers
24 views

Randomized Submatrix of a Sparse Matrix

I have a sparse square matrix $A$ with size $n \times n$ and number of nonzero entries $nnz$. The goal is making a sub-matrix $B$ with $s$ nonzeros which are randomly chosen from $A$. Duplicates are ...
5
votes
2answers
105 views

Fast and Numerically Stable Pairwise Distance Algorithms

I'm looking for resources on fast, numerically stable pairwise euclidean distance algorithms. In particular, suppose $A \in \mathbb{R}^{M \times D}$ and $B \in \mathbb{R}^{N \times D}$ are two sets of ...
2
votes
1answer
83 views

Inverting small matrices: canned factorization versus explicit formula

I am interested in solving a large number of small linear systems of equations, $Ax=b$, with $A$ either $2\times2$ or $3\times3$. Assuming none of these systems are actually singular, is there ...
1
vote
1answer
134 views

Solving linear system with matrix multiplication

When solving a linear system $Ax=b$ where $A=B^TCB$ do I need to form $A$ explicitly by two matrix-matrix multiplications or is there another more simple way? $C$ is a NxN matrix and not always ...
4
votes
1answer
151 views

Do I really need to invert this matrix

I need to calculate a matrix $A$ (at least some elements of it, see below) as defined by the following equation $$ A=B(\mathbb{1}-B)^{-1} $$ where B is a square matrix of dimension $N$ and $\mathbb{...
3
votes
1answer
123 views

Derivatives of Approximate Matrix inverses

I am cross posting this question to the mathermatics stack exchange. please find it either at this link, https://math.stackexchange.com/q/2952989/430980, or below: I have a question concerning the ...
3
votes
3answers
157 views

Derive the formula for eigenvalues

If $A$ has eigenvalue $\lambda_A$ $$B = I - c\frac{I-rA}{I-\bar{r}A}$$ How to derive the eigenvalue $\lambda_B$? $$\lambda_B=1-c\frac{1-r\lambda_A}{1-\bar{r}\lambda_A}$$ where $c, r, \bar{r}$ are ...
8
votes
0answers
174 views

Imbalance of variables in Mixing Newton's method and Linear solver for a Non-linear system

Problem Solving a non-linear system of equations. The number of variables is the same as the number of equations. When I fix a set of variables (say $\vec{y}$) and keep another set free (say $\vec{...
1
vote
1answer
66 views

Pivoting in Block LU

What are common methods to choose pivot blocks in Block LU (for non-SPD/non-Diagonally Dominant Matrices)?
4
votes
2answers
98 views

Term for the typical “linear in the larger dimension, quadratic in the smaller” cost for linear algebra

Many dense linear algebra decompositions (QR, SVD...) on an $m\times n$ matrix have cost $$ O(\max(m,n)\min(m,n)^2) $$ when implemented in practice on a computer. Is there a colloquial name or a more ...
1
vote
0answers
61 views

finding null space to a complex matrix

I need to solve the following equation: $$ \begin{pmatrix} \frac{\omega^2}{c^2}\varepsilon_x-\mu_z^{-1}k_y^2-\mu_y^{-1}k_z^2 & \mu_z^{-1}k_xk_y & \mu_y^{-1}k_xk_z\\ \mu_z^{-1}k_xk_y &\...
2
votes
2answers
115 views

Memory/speed tradeoff for many small matrix inverses

Problem In the case of a finite element code, I have many small (order of 30x30) matrix inverses (or LU factorizations), one per finite element. These matrix inverses never change and must be applied ...
3
votes
2answers
126 views

Moore-Penrose pseudoinverse of singular rank degenerate matrix

I am trying to attain the Moore-Penrose pseudoinverse of a very large, very sparse, rank-degenerate, singular, and square matrix. ($75000 \times 75000$, near rank). The matrix is a graph Laplacian and ...
3
votes
0answers
114 views

Computing Small Eigenvalues with Sparse Symmetric Indefinite Mass Matrix

I want the eigenvalues of the following generalized eigenvalue problem: $$ Av = \lambda M v $$ where $A\in\mathbb{R}^{n\times n}$ is sparse, symmetric, and positive semi-definite $M\in\mathbb{R}^{n\...
1
vote
1answer
65 views

TDMA with 3rd order upwind scheme

I'm trying to implement a model I found in a paper, but there is something I do not understand. The authors say they use TDMA to solve their equations; however, they use a 3rd order upwind biased ...
2
votes
1answer
110 views

How many operations are needed for LAPACK's zgesv to solve a linear system?

I have a linear system of complex numbers. I am using LAPACK' zgesv (actually I am using intel MKL LAPACKE, but I am assuming the algorithm is the same). No assumption can be made about the system. I ...
1
vote
1answer
53 views

Thomas Algorithm Kernel OpenCL

I am trying to implement the Thomas algorithm using OpenCL. ...
2
votes
2answers
178 views

Lost on Matrix Inversion

I try to implement some big matrix inversion. My system configuration is Hardware:- Memory: 62.8GiB, Processor: Intel Xeon(R)CPU E5-2670 v3 @2.30GHZ*48 To implement matrix inversion I am using ...
4
votes
1answer
68 views

Method to Efficiently Solve “Centered” Least Squares without centering “A”

Suppose I want to solve $$\text{arg min}_x \frac{1}{2}\|\tilde{A}x - b\|_2^2 + \frac{1}{2}\|x - c\|_2^2$$ where $A$ is a wide sparse matrix and $\tilde{A} = A C_n = A (I - \mathbf{1}\, \mathbf{1}^T/...
1
vote
0answers
46 views

Methods to compute specific eigenvector components for a tridiagonal matrix

I have an application that is somewhat similar to the situation of computing Gaussian quadrature nodes and weights: simply put, I need to compute the eigenvalues and the last two (normalized) ...
1
vote
1answer
153 views

Ways to solve $Ax=b$ for a sparse (banded) $A$ with updates

I want to solve the time-dependent Schrodinger Equation using the Crank-Nicolson scheme. I end up with the following matrix equation ...