This can happen if Microsoft Visual Studio Visual C++ compiler is installed in a non-default directory. The mex options file looks for MSVC 8.0 in a directory relative to the environment variable VS80COMNTOOLS and MSVC 7.1 in a directory relative to the environment variable VS71COMNTOOLS.
You can find out what this environment variable is set to by using the following command at the MATLAB command prompt
getenv VS80COMNTOOLS
or
getenv VS71COMNTOOLS
If the environment variable for the appropriate compiler does not exist, you will need to either re-install Microsoft Visual Studio or create the environment variable manually. It needs to point to the Common7\Tools\ sub directory of your Microsoft Visual Studio installation. This is often located at:
C:\Program Files\Microsoft Visual Studio 8\Common7\Tools\
or
C:\Program Files\Microsoft Visual Studio .NET 2003\Common7\Tools\
Microsoft Visual C++ should be installed in a directory called “VC” (for MSVC 8.0) or "VC7" (for MSVC 7.1). This directory should be located two directories above the directory specified by VS80COMNTOOLS or VS71COMNTOOLS. If this is not the case, the setup will not be able to locate your compiler automatically. For example if VS80COMNTOOLS is set to:
C:\Program Files\Microsoft Visual Studio 8\Common7\Tools\
then Microsoft Visual C++ is expected to be located at
C:\Program Files\Microsoft Visual Studio 8\VC
You can either re-install Microsoft Visual Studio in this arrangement or do the following (example given for MSVC 8.0):
1. When prompted: "Would you like mex to locate installed compilers [y]/n?" type "n".
2. Select "Microsoft Visual C/C++ version 8.0"
3. When prompted: "Use C:\Program Files\Microsoft Visual Studio 8 anyway [y]/n?" type "n"
4. Enter the correct path to Microsoft Visual Studio, i.e. "C:\MYCOMPILERDIR\Microsoft Visual Studio 8".
