Automatic differentiation allows us to numerically evaluate the derivative of a program on a particular input. There is a theorem that this computation can done at a cost less than five times the cost to run the original program. This factor of five is an upper bound.
In what situations can this cost be further reduced? Many in-the-field derivative codes run at near the speed of the original program. What is done to obtain this speed-up?
What are traits of the original program that can be exploited to speed up computation?
What software engineering tricks can be employed to speed up computation?