Consider an ODE of the form:
$$d_t u(t) = \alpha u(t) + b(t), \quad b(t) = \exp(t\beta ).$$
By using an integrating factor of $\exp(-t\alpha)$ you can get the solution:
\begin{align} u(t) &= \exp(t\alpha)u(0) + \int_{0}^t\exp((t-s)\alpha)b(s)\,ds \\ &= \exp(t\alpha)u(0) + \exp(t\alpha)\int_{0}^t\exp((\beta-\alpha)s)\,ds. \end{align}
Set $\gamma = \beta-\alpha$, then if $\gamma = 0$ the solution is:
\begin{equation} u(t) = \exp(t\alpha)u(0) + \exp(t\alpha)\int_{0}^t\,ds = \exp(t\alpha)(u(0) + |t|) \end{equation}
Now let $\gamma\ne 0$, then:
\begin{align} \int_0^t\exp(\gamma s)\,ds &= \int_0^t\sum_{k=0}^{\infty}\frac{ s^k\gamma^k}{k!}\,ds = \operatorname{sign}(t)\sum_{k=0}^{\infty}\frac{t^{k+1}\gamma^k}{(k+1)!} \\ &=\operatorname{sign}(t)\gamma^{-1}\gamma\sum_{k=0}^{\infty}\frac{t^{k+1}\gamma^k}{(k+1)!} = \operatorname{sign}(t)\gamma^{-1}\sum_{k=1}^{\infty}\frac{t^k\gamma^k}{k!} \\ &= \operatorname{sign}(t)\gamma^{-1}(\exp(t\gamma)-1). \end{align}
Then if I didn't mess up something, the solution is: $$u(t) = \exp(t\alpha)(u(0) - \operatorname{sign}(t)\gamma^{-1}) + \exp(t\beta)\operatorname{sign}(t)\gamma^{-1}.$$$$u(t) = \exp(t\alpha)u(0) +\operatorname{sign}(t)\frac{\exp(t\beta) - \exp(t\alpha)}{\beta - \alpha}.$$
Note that in the limit $\beta \to \alpha$ the second term becomes $\exp(t\alpha)|t|$, so it is consistent with the $\gamma = 0$ case. You don't need to do any time-stepping now, and you can analyze the numerical errors you can get in this expression. You can also use this to check against your numerical solutions, but either will break for a nasty combination of parameters, it's just the time-stepping would probably break earlier. As mentioned in the other answer, your problem is unstable in the backwards direction - things grow exponentially, and your error tends to blow up.