I had the same problem to configure the C++ compiler provided by MSVC 2017 build tools with MATLAB R2018b. The solution was quite simple. If you look inside the file located here:
you will see, that MATLAB uses the entries in the registry to search for installed MSVC compilers. Therefore I only added a text entry to
with the name "15.0" and value "C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools". After adding this entry, restart MATLAB and repeat the configuration of the mex compiler.
You can use the attached file to add this entry to the registry.