Questions tagged [blas]
Basic Linear Algebra Subprograms - A standard API library with vector-vector, matrix-vector, and matrix-matrix operations.
84
questions
4
votes
1
answer
255
views
Matrix Diagonalization and Computational Requirements
I have some questions about diagonalizing matrices. My interest lies in computing all eigenvalues of a given matrix. To avoid wasting time and improve my research efficiency, I want to understand the ...
- 143
0
votes
1
answer
77
views
Weird runtime behavior of `scipy.linalg.solve_triangular` and `trtrs`
I want to understand the time complexity of scipy.linalg.solve_triangular, which calls trtrs from LAPACK under the hood, so I ...
- 181
1
vote
1
answer
242
views
How do BLAS libraries implement support for transposed matrices?
I'm trying to understand how BLAS libraries implement fast GEMM with support for transposed matrices. Say, I'm only operating on square matrixes (with dimensions n ...
- 121
2
votes
1
answer
138
views
What is the correct way of computing $LL^T$ in packed format with blas/lapack
I have a triangular matrix stored in packed format (ie $L$). I need to compute $LL^T$ (not the decomposition, just the multiplication). What would be the preferred way of computing this with blas/...
- 23
11
votes
1
answer
352
views
Is it possible to express an arbitrary tensor contraction in terms of BLAS routines?
I noticed that libraries like numpy and pytorch are able to perform arbitrary tensor contractions at speeds similar to comparably sized matrix multiplications. This leads me to believe that underneath ...
- 121
2
votes
1
answer
186
views
Fast weighted vector inner product x*A*y with BLAS/LAPACK
Is there a way to compute the weighted vector inner product xAy with vectors x and y and Matrix A using BLAS/LAPACK while avoiding additional allocations or overwriting the inputs?
I'm happy with ...
- 799
2
votes
1
answer
205
views
Using Sundials CVODE in MATLAB
I'm currently using ode15s to solve a set of stiff differential equations.
I am trying to use the MATLAB profiler to understand the section of the ode solver code which calls BLAS routines.
Since the ...
- 433
1
vote
1
answer
97
views
Automatic differentiation (AD) of a loss function which maps unitary matrix onto number
Is it possible to estimate whether automatic differentiation (AD) techniques could enable a more efficient way to repeatedly compute the derivative $\delta L / \delta u^*_{ij}$ of a specific loss ...
- 111
2
votes
1
answer
236
views
Parallelize pseudo inverse of a matrix using Lapacke
I am currently using the protocol described in
https://stackoverflow.com/questions/55599950/computation-of-pseidoinverse-with-svd-in-c-using-blas-and-lapacke to compute the pseudo inverse of a matrix.
...
3
votes
0
answers
139
views
Compute orthogonal complement using BLAS / LAPACK
Is there a fast method to compute an orthogonal complement of an arbitrary matrix $U\in\mathbb{R}^{m \times n}$ in BLAS / LAPACK?
Specifically, I want any matrix $V\in \mathbb{R}^{m \times (m - \text{...
- 799
0
votes
1
answer
219
views
How to efficiently transpose distributed matrix in Scalapack?
I have a distributed matrix in block cyclic layout. Is there an efficient way to out/in place transpose a distributed matrix with scalapack?
Context: I am trying to diagonalize the transpose of a ...
- 103
1
vote
0
answers
142
views
Does cblas_dgemm mutate my input matrices?
I have written a matrix class Matrix<T> for which I have implemented a wrapper function for cblas_dgemm.
...
- 111
-1
votes
1
answer
293
views
Armadillo BLAS Matrix Multiplication with it transpose. Blas is too slow?
Does someone knows another trick or solution how can I perform matrix multiplication by its transpose?
The current code for 1000 iterations takes too much time for me.
...
5
votes
2
answers
614
views
Getting to know about various BLAS implementations
I keep coming across phrases like "highly optimized BLAS kernels" with "architecture-specific optimizations", but have never been able to find what exactly these optimizations are, ...
- 161
1
vote
2
answers
1k
views
How to set up the differential equation system to speed up computation?
I've set up a system of differential equations, obtained after discretizing pde, in the following way
...
- 433
2
votes
1
answer
208
views
Efficient change of basis real positive definite symmetric matrix
I need to optimize a code where the most performance critical part is doing a 'change of basis', in other words it is an unitary similarity transformation on a big real positive definite symmetric ...
1
vote
1
answer
242
views
Functions from Scipy, Blas, or Lapack that compute only upper triangular matrix
My goal is to transform a matrix into upper triangular form in Python. I know the function scipy.linalg.lu will do LU decomposition and get both upper and lower ...
- 207
4
votes
0
answers
825
views
What algorithm do BLAS and ATLAS use for matrix multiplication?
I have searched and what I understood was that they use the naive one with several memory and cache optimizations.
However, I wanted to know whether they are using the Strassen or the Coppersmith-...
- 51
1
vote
1
answer
148
views
Using LAPACK to compute $B^{-1}AB^{-T}$ for thin $B$
How can I use BLAS/LAPACK to compute
$$
B^{-1}AB^{-T}
$$
where $A\in\mathbb{R}^{n,n}$, $B\in\mathbb{R}^{m,n}$ is full rank matrix with $m>n$, and $B^{-1}y:=\arg \min_{x} \|Bx-y\|_{2}$.
In theory, ...
- 799
7
votes
0
answers
632
views
Is there any catch on using `zgemm3m` vs regular `zgemm`?
I've just (to my embarrassment) encountered a BLAS-like extension of a matrix-matrix product subroutine gemm in Intel MKL: gemm3m...
- 8,712
2
votes
0
answers
47
views
Best way of porting code from the GPU to MPI-nodes
I have a program, structured in two parts, $A$ and $B$. Both parts are capable of running as standalone units, and written in C++. $A$ is written for cluster systems, running entirely on CPU-nodes, ...
- 563
2
votes
1
answer
214
views
BLAS operation question
I want to perform the following operation:
$$
A = A + U B^T
$$
where $A$ is $m \times n$ dense, $U$ is $m \times m$ upper triangular, and $B$ is $n \times m$ dense. The BLAS function ...
- 1,078
2
votes
3
answers
267
views
C standard for computational science
Which C standard should be used for computational science code ?
Should we keep compatibility with C89/90/ANSI or jump to C99 or C11 ?
Context:
Code will use third-party : BLAS, LAPACK, MKL, ...
4
votes
0
answers
108
views
Block matrix and DSYRK
I want to compute the matrix
$$
A = \sum_{i=1}^N v_i v_i^T
$$
where each $v_i$ is a given vector of length $2500$, so that $A$ is $2500 \times 2500$, and my $N$ is about 2 million. Rather than call ...
- 1,078
3
votes
2
answers
240
views
Parallel assembly of matrix
I have a matrix which I want to assembly quickly, which is in block form:
$$
A = \pmatrix{ A_{11} & A_{12} & A_{13} \\ A_{21} & A_{22} & A_{23} \\ A_{31} & A_{32} & A_{33}}
$$
...
- 1,078
4
votes
1
answer
337
views
Fast matrix multiplication with matrix elements computed on-the-fly (without forming the matrix)
Is there any library or routine for high-performance matrix-matrix product, where the matrix elements are computed on-the-fly using a given function of $i$ and $j$?
More specifically, in the problem ...
- 141
1
vote
0
answers
89
views
Fast calculation of $A^T B$
I need to compute a matrix-matrix product, $A^T B$, where $A$ is $n \times r$ sparse, and $B$ is $n \times q$ dense. The number of rows $n$ is far larger than both $r$ and $q$. In fact $n$ is so large ...
- 1,078
1
vote
1
answer
273
views
Wrong result of 'ddot' from BLAS
I am having trouble with a C/C++ program that uses the BLAS routine ddot.
I am running Linux and so far LAPACK routines worked without any problems.
I get a wrong ...
- 25
1
vote
0
answers
159
views
$ A * B $ computation when B is a symmetric matrix in armadillo [closed]
Is there any way to multiply a symmetric matrix by a dense one in armadillo(and use the fact that we have a symmetric matrix)? I know about DSYMM Routine in BLAS,...
- 131
1
vote
0
answers
312
views
Kronecker product of matrices
I have to use Kronecker product of a matrix with a unit matrix is there any routine in ScaLapack or Lapack which can do so efficiently.
1
vote
0
answers
128
views
Eigenvalue problem (LAPACK)
I am working on a project in numerical analysis which I have to program in C (using Lapack and Blas). Matrix is given which is tridiagonal and "almost" symmetric (one element is to be changed to make ...
- 111
2
votes
1
answer
1k
views
The difference between mkl_intel_lp64 vs mkl_gf_lp64 in a numerical reproducibility issue with Intel MKL
It has been discussed that Intel MKL can exhibit irreproducible behavior under certain conditions. In fact, this is a known thing and described by Intel as Conditional Numerical Reproducibility.
A ...
- 143
1
vote
1
answer
195
views
GPU libraries for integer matmul | overflow tolerated
Are there any high performance integer BLAS libraries that implement matrix multiplication i.e. i32gemm and i64gemm ? I need to use them for a cryptographic application and can tolerate overflows, i.e....
- 287
2
votes
0
answers
205
views
What matrix criteria is "large enough" for Eigen to use a BLAS backend?
Eigen's documentation says that any BLAS back-end can be used (e.g. MKL) to perform the actual matrix calculations.
It is, however, very vague when discussing the criteria as to which the ...
- 802
2
votes
1
answer
771
views
Distributed (MPI) matrix matrix multiplication
I perform matrix matrix multiplications (between rank-3 and rank-2 arrays) in fortran using following subroutine,
...
- 125
17
votes
3
answers
2k
views
Are BLAS implementations guaranteed to give the exact same result?
Given two different BLAS implementations, can we expect that they make the exact same floating point computations and return the same results? Or can it happen, for instance, that one computes a ...
- 12k
3
votes
2
answers
7k
views
BLAS libraries for Octave or Matlab, preferrably with GPU support?
I just searched around a bit for BLAS implementations and was amazed by the sheer amount of libraries around. Does someone know of a benchmark or otherwise rating of the various libraries?
How easy ...
- 380
1
vote
1
answer
124
views
Set strictly upper triangular part of a matrix to alpha using BLAS or LAPACK
Is there routine in standard BLAS or LAPACK to set strictly-upper triangular part (the part above the diagonal) of a matrix to alpha?
I do not want to change diagonal elements so ...
- 273
6
votes
4
answers
8k
views
Beating typical BLAS libraries matrix multiplication performance
A dull matrix multiplication algorithm where we use the formula
$$C_{ij}=\sum_{k}A_{ik}B_{kj}$$
By literally following this in 3 loops we'll get a very slow program, because we don't utilize ...
5
votes
2
answers
1k
views
Smart way to multiply 3 matrices
I have a quantum mechanics simulation where I need to multiply three matrices that look like this:
$$\rho(t_1)=U^\dagger \rho(t_0) \, U$$
where $U^\dagger$ is the hermitian conjugate of $U$. This ...
2
votes
2
answers
4k
views
Fastest way to perform element-wise multiplication on a sparse matrix
I have two large-ish matrices (~100K cols x ~100K rows).
They are sparse and symmetrical (about 0.1% of them values are non-zero).
I want to do element-wise multiplication between them.
Also, I ...
- 123
7
votes
0
answers
932
views
Sparse matrix format and sparse-matrix sparse-matrix multiplication
I'm having some performance problems with my code dealing with the multiplication of big sparse matrices (stiffness and aerodynamic influence coefficient matrices).
Mainly I have to multiply such ...
- 71
1
vote
1
answer
543
views
Efficiently rotate vector in 2D (and 3D)
I need to efficiently rotate a 2D (and 3D) vector in a CUDA kernel. I was thinking about generating random unitary rotation matrices. I don't need to know the angle, it just has to be randomly ...
- 13
12
votes
0
answers
4k
views
Optimized open source BLAS / LAPACK package
I was wondering what is a more optimized open source BLAS/LAPACK package with respect to modern multi-core processors (Haswell and beyond). Is there any distribution that can attain performance close ...
- 121
4
votes
2
answers
455
views
Optimized parallel routine for $X' W X$ with $W$ diagonal
$X$ is a dense matrix of real doubles, typically of size 20 million rows and 500 columns, and $W$ is a diagonal matrix of real, non-negative doubles stored as a vector. I'm working in C and have ...
- 141
3
votes
2
answers
97
views
Do BLAS routines compute their respective operations with minimum error?
Do all BLAS routines compute the respective operation with minimum error ?
i.e. Is the reduction in sdot computed with least error ?
I need to call these ...
- 287
1
vote
1
answer
406
views
Kronecker products and basis contractions (ie. B.A.Transpose[B]) in C?
I have implemented a basis transformation in C of the following form
kron[A,A]*B*Transpose[kron[A,A]]
where A and ...
- 11
2
votes
0
answers
357
views
Sparse Linear Algebra vs Dense Linear Algebra
I am interested in a reference in the literature that discusses the performance of Dense Linear Algebra (blas routines) and dense linear algebra (sparse blas routines).
I am interested in knowing for ...
- 1,000
3
votes
2
answers
5k
views
BLAS, LAPACK or ATLAS for Matrix Multiplication in C
I am trying to find the most optimized way to perform Matrix Multiplication of very large sizes in C language and under Windows 7 or Ubuntu 14.04. And searching led me to BLAS, LAPACK and ATLAS. ...
- 31
1
vote
1
answer
128
views
How to make this matrix efficiently?
Originally posted on stats.stackexchange, I'll pair the post down to something a bit more general.
Suppose I have vectors $\{\mathbf{\delta}, \mathbf{x}_1, \ldots, \mathbf{x}_J\}$, where $\delta \in \...
- 111