Starting with R2012b MATLAB is fully supported on Mountain Lion and does not require X11 or XQuartz.
For more information, please refer to the platform roadmap:
MATLAB R2012a and earlier versions of MATLAB were developed and released before the availability of Mac Mountain Lion, so compatibility cannot be guaranteed. However, these versions are known to generally work.
If you face issues starting up or using MATLAB on Mac OS X 10.8, please install X11 on your computer by following these steps:
1) Start MATLAB by double clicking on the MATLAB application icon. You will be prompted to install X11 in a window that looks like the following:
2) Click on “Continue”. This will take you to an Apple support page, which will direct you to download and install the DMG file for XQuartz. The XQuartz is an X11 server. Please install the XQuartx version 2.7.2 or higher from this download link. If the apple support page above does not open up, please use the following link to open it:
3) At the end of the installation, you will be reminded to log out and log back in to your Mac. To do so, click on the Apple icon on top-left corner of your Desktop screen, and select "Log Out [username]...". It is important that you do so. Otherwise MATLAB will likely crash upon start-up: