First of all, the term $\frac{\| A x \|}{\| x \|}$ is not constant.
For example, consider a $2 \times 2$ matrix and let $w_1$ and $w_2$ be the columns of $A$. Then,
$$ A x = x_1 w_1 + x_2 w_2 . $$
Now, observe that for $e_1 = (1, 0)^T$ we have that
$$\frac{\| A e_1 \|}{\| x \|} = \frac{\| w_1 \|}{\| x \|} = \| w_1 \|$$
and for $e_2 = (0,1)^T$
$$\frac{\| A e_2 \|}{\| x \|} = \frac{\| w_2 \|}{\| x \|} = \| w_2 \| \,.$$
Obviously, $\| w_1 \|$ and $\| w_2 \|$ do not need to be the same.
Now, assume that you want to compute $A e_1$; the solution is $w_1$. Furthermore, let $\| w_2 \| \gg \| w_1 \|$.
On a computer, however, we are in general not able to perform an exact computation; we have to deal with round-off errors. Hence, consider the perturbed matrix vector product,
$$ A (e_1 + \epsilon e_2) = w_1 + \epsilon w_2 \,. $$
The difference between the true solution and the perturbed one, which is
$$ w_1 + \epsilon w_2 - w_1 = \epsilon w_2 \,, $$
is large (in comparison to the size of $w_1$) even if $\epsilon$ is small, since $\| w_2 \|$ is large. Hence, the value of the perturbed matrix vector product is very different from the true value, while the perturbation of the input data was small.
You can see that it can cause problems, when a matrix sends vectors of the same magnitude to vectors of very different magnitudes, and the quotient of the largest and smallest possible magnitude is exactly the definition of the condition number.
The condition number of a matrix appears in many different contexts. One can, e.g., imagine that a large condition number also causes trouble, when solving a linear system. Take a look at @GertVdE great answer for that matter.