I think it depends on what you are doing. I do typically mix languages. I'll describe my personal experience below, perhaps this'll help you decide for your application. While I'll be talking about my own way of working below, and my tool of choice is Mathematica, I believe the same advantages apply to other high level languages too.
I typically write some type of simulation in a low level (C++) language. It has some (numerical) input and produces some potentially complex numerical output. Originally I passed the input through command line parameters and input files, and wrote the output into text files.
Eventually it became necessary to run the program for many different input parameters, which is best automated. I used to use some mix or shell scripting and scripting languages (Python) do call my program with various parameters / input-files, and processed/visualized the output in Mathematica.
Eventually I figured out that it's much better to write a Mathematica interface for calling my C++ code instead of writing a command line interface and using input files. Now this is my standard workflow. The Mathematica interface turned out to be simpler and easier to write (once I learned how!), mainly because of how easy it is to transfer structured numerical data without the need to convert it to text first. Consider e.g. having to pass a list of 3D arrays. One would need to invent a special storage format to transfer these, or use a C++ library for some common data format such as HDF5, which Mathematica already has a high level interface for anyway.
Using a Mathematica interface also made a number of things very easy, such as:
smarter mapping of the parameter space (e.g. applying various adaptive sampling algorithms, which are much easier to implement and test in a high level language);
easily save the simulation state and resume later (because transferring structured numerical data between Mathematica and C is easy enough that I can afford to take the time to implement this)
run various numerical algorithms with my simulation, such as multivariate optimization, root finding, etc. (because Mathematica already has many of these built in)
Visualizing the results immediately and interactively, without having to go through several steps (write the data to a file, process it for visualization, read it into the plotting program). Suppose that a simple root finding method would fail, and it's not clear why. If I can play with the simulation interactively, and visualize results immediately, it's much easier to figure out what's going on.
Easily running simulations in parallel using Mathematica's parallel tools, even on multiple computers (thanks to Mma's built-in communication protocol).
If the output of the simulation is very large (hundreds of MB), I can't afford to store it all for all parameter values. I'd usually run some analysis on it (averages, variances, etc.), and store only the results. Using the high level language to drive the simulation allows me to run many more non-trivial types of analyses without having to implement them myself in C++.
The focus here is on the ease of implementing things using a high level language with lots of useful built-in functionality. All of this could be done differently too, but it would be much more work, so I likely wouldn't do it.
Now I typically write the Mathematica interface before writing a command line interface, and only do the latter if I actually need it (e.g. I have to share the code with people who don't use Mathematica).
This worked well for me because I already knew Mathematica well, and I already used it for processing and visualizing the output anyway. I am also familiar with Mathematica's convenient C interface, which does take some time to learn. Also, I usually write these simulations alone, which doesn't restrict the tools I can choose. If I were collaborating with someone on the code, I would choose to use tools that we both know well.
The same advantages apply when using any other similar high level language with a convenient C interface, such as Python/MATLAB/Julia/R/etc. I'm mentioning Mathematica here because that's what I am familiar with and I'm describing my own workflow.
I also regularly use MATLAB and R functions (and occasionally Java) from within Mathematica. I do this because convenient high level interfaces already exist between these languages. Otherwise I wouldn't bother. I believe Python is one of the most convenient languages to use in this manner, gluing together functionality from various packages.