MATLAB tailored engineering Benchmarks for AI Models?
Hello everyone,
Today, we have a wide range of models available for problem-solving, education, and agentic coding. Picking a model for a task itself is a skill worth learning and exploring. We have a bouquet of closed models from Anthropic, OpenAI, Grok (Cursor), Mistral, and Gemini, as well as open-source models like Kimi and Gemma.
I am wondering whether, as a MathWorks community, we can come up with some benchmarks tailored for MATLAB + AI models by choosing and compiling tasks like plotting (graphing), some matrix problems, signal processing tasks, control system problems and other engineering problems, and then test AI model with Matlab-MCP server and document performance.
Reaching a consensus would really help us choose a certain model for certain tasks. That way, we can save API tokens, be smart with our choices and as a community, we can learn and help by sharing our experiences. I would love to hear your thoughts and see what this conversation could lead to.
@Hans Scharler do you have any initial thoughts to share with us?
Tags
