I've found a solution towards this with the help of Matlab Support team. Thank you!
In my set up, I need to expose head and worker externally so that my client UI, outside of the VPC could connect with them. However, Matlab Parallel Server has an assumption that communication between machines within the cluster is not restricted, which implicates all ports need to be open for woker-worker communication.
The solution is to add
in the mjs_def file, with the hostname in that file to be external IP. In this case, all worker-worker communication is no longer through external Load Balancer service, but locally. (In my set up, all worker workloads are within the same GKE cluster so they could talk internally)