As @FedericoPoloni commented on the post above, a natural way to expedite this search (relative to simple descending trial division) is by Fermat's Factorization Algorithm. As the Wikipedia article illustrates, the Fermat approach can take bigger steps than trial division, but at a cost of more arithmetic (square roots in extended precision versus only integer division).
Fermat's idea is to search for a difference of integers squares $N = a^2 - b^2$, so that $N = (a + b)(a - b)$. If the search starts with $a \ge \sqrt N$, then incrementally increasing $a$ finds the first such pair of "cofactors" of $N$, $c = a+b, d = a-b$ as close together as possible. CAVEAT: The cofactor pair $c,d$ would then have the same parity!
This means that we may either (i) not be able to express $N$ as a difference of two integer squares (in the case that $N = 4k+2$) or (ii) find a pair of same parity cofactors which are not the desired closest together pair of cofactors. An example of the first problem is $N=6$, which is not a difference of integer squares as shown in answers to this older Math.SE Question.
An example of the second problem is $N=12$, which can be expressed as a difference of squares $12 = 4^2 - 2^2 = 6\cdot 2$, but whose closest cofactors are instead $12 = 4\cdot 3$ (and not of the same parity).
Of course if $N$ is odd, it has only odd divisors, so any two cofactors will be the same parity (odd). To handle the case when $N$ is even, one can apply Fermat's method to $4N$. Now if we locate a cofactor pair of the same parity, they will both be even! So we will have $4N = (2c)(2d)$, and hence $N = cd$. Furthermore our search will locate the closest same parity cofactors $2c,2d$ of $4N$. It follows that $c,d$ are the closest cofactor pair of $N$ (though not necessarily of the same parity).
If $N$ is odd, then while a correct result will obtain from applying Fermat to $4N$, there is an advantage to applying directly to $N$. The oddness of $N$ means that numbers of the form $4k+1$ will be expressed as an odd square minus an even square, so our search is limited to checking $a$ odd. On the other hand numbers of the form $4k-1$ will be an even square minus an odd square, and again our search is limited (to checking $a$ even).
Algorithm: Fermat's Factorization Method (FFM)
Given as input positive integer $N \ge 1$ which is odd or a multiple of $4$:
Initialize integer $a = \lceil \sqrt N \rceil$. If $a = \sqrt N$, stop. OUTPUT: The closest cofactors are $c = d = \sqrt N$. Otherwise:
Now $a^2 > N$, so compute $a^2 - N$ and $b = \lfloor \sqrt{a^2 - N} \rfloor$. If $b = \sqrt{a^2 - N}$, stop. OUTPUT: The closest (same parity) cofactors are $c = a + b, d = a - b$. Otherwise:
Increment $a$ suitably (parity consideration) and go to Step 1.
It is possible to analyze at what point trial division becomes as efficient as Fermat's method, due to its trade-off of step size versus complexity. I plan to do an extended precision implementation, with the speculation that the time required for Fermat's step is about three times the time needed for trial division.