This really depends on the operations you are including in your question. If you took the sparse equivalent of any level 1 BLAS or level 2 BLAS algorithm, then yes they are memory bound (not compute bound), but that is also true of level 1 BLAS and level 2 BLAS for dense matrices. To answer this we need to understand what makes an algorithm compute bound.
Compute bound algorithms are those for which their performance relies largely on the capability of the underlying system to perform arithmetic, often measured in FLOPS.
For an algorithm to have this property it must do significantly more arithmetic operations than memory operations, because memory operations have very high latency. In the context of linear algebra algorithms this means there must be significant data re-use in the course of the algorithm. Data re-use means we can avoid resorting to main memory. Level 3 BLAS operations often have this property.
For example to do a sparse matrix-vector product you scan through its list of nonzeros only once. There is no way to cache any values in any way that could help performance. The same is true of dense matrix-vector product for that matter. These are not examples of compute bound algorithms.
Unfortunately most compute architectures today are very good at arithmetic and only decent at memory performance. Thus A lot of sparse linear algebra algorithms are specifically designed to reduce to dense linear algebra algorithms from level 3 BLAS at its most demanding points. Multifrontal factorization is an example of an algorithm like this.
So the overall answer to you is that: Sparse linear algebra is memory bound for many operations, but so is dense linear algebra. We can choose/design our algorithms to match the capabilities of the system at hand however, so this is not necessarily a bad thing.