This post uses the same notation and preliminaries as an answer I wrote to an earlier post.
Let's again suppose that you're integrating over a smooth manifold $\mathcal{M}$, and that you have a diffeomorphism $\varphi: [-1,1]^{2} \rightarrow \mathcal{M}$ that maps $[-1,1]^{2}$ to your manifold. Since $\varphi$ is a diffeomorphism, it has the following properties:
- it is continuously differentiable
- it is invertible
- its inverse is continuously differentiable
We will need all of these properties.
I'm going to assume that $H_{1}$ is a scalar-valued function, and for simplicity of notation, I'm going to drop the subscript and refer to it as $H$.
Let's suppose further that the arguments $(e,n)$ refer to coordinates in $[-1,1]^{2}$, and the coordinates $(x,y,z)$ refer to points in $\mathbb{R}^{3}$, where $\mathcal{M} \subset \mathbb{R}^{3}$.
From your question, it looks like you have $H: [-1,1]^{2} \rightarrow \mathbb{R}$, which is to say that the function you've stated is given in terms of the arguments $(e,n)$. We'll assume that $H$ is continuously differentiable.
Let $G: \mathcal{M} \rightarrow \mathbb{R}$ be defined by $G = H \circ \varphi^{-1}$, or $G(x,y,z) = H(\varphi^{-1}(x,y,z))$. We're using $\varphi^{-1}$ as a "change of coordinates" here, mapping $(x,y,z)$ coordinates of points in $\mathcal{M}$ to $(e,n)$ coordinates of points in $[-1,1]^{2}$.
It doesn't make sense to differentiate $H$ with respect to $(x,y,z)$, but it does make sense to differentiate $G$ with respect to $(x,y,z)$. I think the integral you want to express above is:
\begin{align*}
\int_{\mathcal{M}} \left(\frac{\partial{G}}{\partial{x}}\right)^{2}(x,y,z) + \left(\frac{\partial{G}}{\partial{y}}\right)^{2}(x,y,z) + \left(\frac{\partial{G}}{\partial{z}}\right)^{2}(x,y,z)\,\mathrm{d}S,
\end{align*}
more compactly expressed as:
\begin{align*}
\int_{\mathcal{M}} \|\mathrm{D}G^{T}(x,y,z)\|^{2}\,\mathrm{d}S = \int_{\mathcal{M}} \|\nabla G(x,y,z)\|^{2}\,\mathrm{d}S.
\end{align*}
By the chain rule,
\begin{align*}
\mathrm{D}G(x,y,z) = (\mathrm{D}H \circ \varphi^{-1})(x,y,z) \cdot \mathrm{D}(\varphi^{-1})(x,y,z),
\end{align*}
where the $\cdot$ denotes matrix multiplication.
A sanity check is in order here:
$\mathrm{D}H$ is a 1 by 2 matrix (the Jacobian of a scalar function with respect to multiple variables is a row matrix)
$\mathrm{D}(\varphi^{-1})$ is a 2 by 3 matrix
$\mathrm{D}G$ is a 1 by 3 matrix
the right hand side of the equation is a 1 by 3 matrix (the product of a 1 by 2 matrix and a 2 by 3 matrix
Given all of this information (it's quite a long post already), you should be able to use my answer to your previous post to calculate your integral. I can fill in the additional steps, if need be.
You make an excellent point in your question that:
The derivatives of the first shape function with respect to $(x,y,z)$ are not the same as the derivatives with respect to $(e,n)$. I should use the inverse Jacobian matrix to translate the values of the derivatives with respect to $(e,n)$ at the integration point to the derivatives with respect to $(x,y,z)$.
If $f: X \rightarrow X$, where $X \subset \mathbb{R}^{n}$ for some natural number $n$, and $f$ is a diffeomorphism (again, continuously differentiable function with continuously differentiable inverse), it is true by the chain rule that $\mathrm{D}(f^{-1}) = \mathrm{D}f^{-1}$. The problem with this concept for your problem (as you rightly point out) is that $\varphi$ is a diffeomorphism, but doesn't have the same number of input arguments as output values, so we can't state that $\mathrm{D}(\varphi^{-1}) = \mathrm{D}\varphi^{-1}$ unless we interpret the "inverse" of the nonsquare matrix as some sort of generalized matrix inverse. I hope it is clear from the reasoning above how you get from the "inverse matrix" idea to the more general case.