Thread Subject:

"Segmentation violation" and "Abort signal" fatal errors related to MEX file

Hello all—

I wrote a (at least, I think) fairly simple MEX file to speed up arburg. Everything was great, it sped up everything wonderfully, got the right numbers...except, when I run my new function, every so often (sometimes after 3 calls, sometimes after 300) Matlab just completely fails, saying there was either a "Segmentation violation" or an "Abort signal" (it's completely random).

So I made a project for the file in Xcode (I'm on 64-bit OS X, 10.8, Matlab R2012a), built it that way, and attached the LLVM debugger to Matlab so I could figure out what was going on. And a few absolutely infuriating things popped up:

1) The error isn't consistent. Sometimes it's an EXC_BAD_ACCES, sometimes it's a SIGABRT. There's no pattern.
2) The error is never triggered inside of my code. I always see the error triggered inside of the bytecode (which is AWESOME to debug, by the way). So I have no idea what part of my code is leading to the problem.
3) I wrote a simple main function to test the same calculations outside of Matlab, and it works flawlessly (has run 1e6 times now without an error).
4) Sometimes the error would occur a Long time After I had called my MEX function. I would call it, be satisfied there was no error, go about my business, and while I was doing something completely different, Matlab would hang in the same way.

It's really the most insane and unnerving thing I've ever seen. I would appreciate any help.

Thank you,

Maxwell J. Collard
Johns Hopkins University

"Maxwell Collard" wrote in message <jviaa6$jiq$1@newscl01ah.mathworks.com>...
> Hello all—
>
> I wrote a (at least, I think) fairly simple MEX file to speed up arburg. Everything was great, it sped up everything wonderfully, got the right numbers...except, when I run my new function, every so often (sometimes after 3 calls, sometimes after 300) Matlab just completely fails, saying there was either a "Segmentation violation" or an "Abort signal" (it's completely random).
>
> So I made a project for the file in Xcode (I'm on 64-bit OS X, 10.8, Matlab R2012a), built it that way, and attached the LLVM debugger to Matlab so I could figure out what was going on. And a few absolutely infuriating things popped up:
>
> 1) The error isn't consistent. Sometimes it's an EXC_BAD_ACCES, sometimes it's a SIGABRT. There's no pattern.
> 2) The error is never triggered inside of my code. I always see the error triggered inside of the bytecode (which is AWESOME to debug, by the way). So I have no idea what part of my code is leading to the problem.
> 3) I wrote a simple main function to test the same calculations outside of Matlab, and it works flawlessly (has run 1e6 times now without an error).
> 4) Sometimes the error would occur a Long time After I had called my MEX function. I would call it, be satisfied there was no error, go about my business, and while I was doing something completely different, Matlab would hang in the same way.
>
> It's really the most insane and unnerving thing I've ever seen. I would appreciate any help.
>
> Thank you,
>
> Maxwell J. Collard
> Johns Hopkins University

Is your mex file short enough to post? We really have not much to comment on yet.

James Tursa

"James Tursa" wrote in message <jviisk$hre$1@newscl01ah.mathworks.com>...
>
> Is your mex file short enough to post? We really have not much to comment on yet.
>
> James Tursa

Sure, no problem. Here it is:


//--------------------------------------//

#include "mex.h"
#include <stdlib.h>

/*
 * fast_arburg.c - port of arburg function
 */

void fast_arburg ( double a[], double E[], double k[], double x[], const int nx, const int p )
{
    int i = 0;
    int j = 0;
    int m = 0;
    
    const int na = p + 1;
    
    // Allocate intermediate arrays.
    double* aprev = (double*) calloc(na, sizeof(double));
    
    double* ef = (double*) calloc(N, sizeof(double));
    double* efp = (double*) calloc(N, sizeof(double));
    double* eb = (double*) calloc(N, sizeof(double));
    double* ebp = (double*) calloc(N, sizeof(double));
    
    if (aprev == NULL || ef == NULL || efp == NULL || eb == NULL || ebp == NULL)
        mexPrintf("Problem allocating intermediate arrays.");
    
    // Initialize a and aprev
    a[0] = 1.0;
    aprev[0] = a[0];
    for (i = 1; i < p + 1; i++)
    {
        a[i] = 0.0;
        aprev[i] = a[i];
    }
    
    // N = length(x);
    const int N = nx;
    double Nd = N; // For double math
    
    // Initialize E
    E[0] = 0.0;
    
    // Initialize intermediates
    for (i = 0; i< N; i++)
    {
        // ef = x;
        ef[i] = x[i];
        efp[i] = ef[i];
        // eb = x;
        eb[i] = x[i];
        ebp[i] = eb[i];
        // E = x' * x ./ N;
        E[0] += (x[i] * x[i]) / Nd;
    }
    
    // k = zeros(1, p);
    for (i = 0; i < p; i++)
        k[i] = 0.0;
    
    // for m = 1:p
    for (m = 0; m < p; m ++)
    {
        // efp = ef(2:end);
        int efp_start = m + 1;
        // ebp = eb(1:end-1);
        int ebp_end = N - (m + 1);
        
        double num = 0.0, den = 0.0;
        for (i = efp_start, j = 0; i < N && j < ebp_end; i ++, j++)
        {
            // efp = ef(2:end); (2)
            efp[i] = ef[i];
            // ebp = eb(1:end-1); (2)
            ebp[j] = eb[j];
            
            // num = -2 .* ebp' * efp;
            num += -2.0 * efp[i] * ebp[j];
            // den = efp' * efp + ebp' * ebp;
            den += (efp[i] * efp[i]) + (ebp[j] * ebp[j]);
        }
        
        // k(m) = num ./ den;
        k[m] = num / den;
        
        for (i = efp_start, j = 0; i < N; i ++, j++)
        {
            // ef = efp + k(m) * ebp;
            ef[i] = efp[i] + k[m] * ebp[j];
            // eb = ebp + k(m) * efp;
            eb[j] = ebp[j] + k[m] * efp[i];
        }
        
        for (i = 1; i < na; i ++)
            aprev[i] = a[i];
        
        // a = [a;0]...
        a[m + 1] = 0.0;
        
        for (i = 1, j = m; i < (m + 2); i++, j--)
        {
            // a = [a;0] + k(m) * [0;conj(flipud(a))];
            a[i] += k[m] * aprev[j];
        }
        
        E[0] *= (1.0 - (k[m] * k[m]));
    }
    
    // Free memory.
    free(aprev);
    
    free(ef);
    free(efp);
    free(eb);
    free(ebp);
}

void mexFunction ( int nlhs, mxArray *plhs[],
                   int nrhs, const mxArray *prhs[] )
{
    double* a; double* E; double* k;
    double* x; double* p;
    int pi;
    
    int nx;
    
    size_t x_m, x_n;
    size_t a_m, a_n;
    size_t E_m, E_n;
    size_t k_m, k_n;
  
    /* Check for proper number of arguments. */
    if ( nrhs != 2 )
    {
        mexErrMsgIdAndTxt( "fastburg:fast_arburg:invalidNumInputs",
            "Two inputs required." );
    } else if ( nlhs > 3 )
    {
        mexErrMsgIdAndTxt( "fastburg:fast_arburg:maxlhs",
            "Too many output arguments." );
    }
    
    // Parameter order:
    // [a, E, k] = fast_arburg(x, p);
    
    /* Assign pointers to each input. */
    x = mxGetPr(prhs[0]);
    
    p = mxGetPr(prhs[1]);
    pi = (int)(*p);
    
    /* Get input and output sizes. */
    x_m = mxGetM(prhs[0]);
    x_n = mxGetN(prhs[0]);
    
    if (!(x_m == 1 || x_n == 1))
    {
        mexErrMsgIdAndTxt("fastburg:fast_arburg:invalidSize",
                          "x must be a column or row vector.");
    }
    if (x_n > x_m)
        nx = (int)x_n;
    else
        nx = (int)x_m;
    
    // a: 1 x (p + 1)
    a_m = (size_t)1;
    a_n = (size_t)(pi + 1);
    // e: scalar
    E_m = (size_t)1;
    E_n = (size_t)1;
    // k: p x 1
    k_m = (size_t)pi;
    k_n = (size_t)1;
    
    /* Create matrix for the return arguments. */
    plhs[0] = mxCreateDoubleMatrix((mwSize)a_m, (mwSize)a_n, mxREAL); // a
    plhs[1] = mxCreateDoubleMatrix((mwSize)E_m, (mwSize)E_n, mxREAL); // E
    plhs[2] = mxCreateDoubleMatrix((mwSize)k_m, (mwSize)k_n, mxREAL); // k
    
    a = mxGetPr(plhs[0]);
    E = mxGetPr(plhs[1]);
    k = mxGetPr(plhs[2]);
    
    /* Call the fast_arburg subroutine. */
    fast_arburg(a, E, k, x, nx, pi);
}

Hi,

I was wondering if this problem have been resolved. I'm having the exact same problem (some what). On my mac laptop, things run flawlessly but when I put it on a linux based cluster (but serial), after so many calls of A\b, MATLAB tells me that it segfaulted.

A is a sparse matrix that is created from a MEX function. I've experienced MATLAB segfaulting when performing operations such as sparse(A) or A' (transpose) when I fill the index wrongly within the MEXfunction. However, when MATLAB segfaults by calling A\b, I check prior to the A\b call if A was filled incorrectly by calling A' and sparse(A) and it passes just fine.

A\b call can segfault after 10 calls, or after 1000 calls. There doesn't seem to be any pattern and I have no idea what's causing it.

Thanks.

-Will

"William Taitano" <williamtaitano1208@hotmail.com> wrote in message <k4ngdk$725$1@newscl01ah.mathworks.com>...
> Hi,
>
> I was wondering if this problem have been resolved. I'm having the exact same problem (some what). On my mac laptop, things run flawlessly but when I put it on a linux based cluster (but serial), after so many calls of A\b, MATLAB tells me that it segfaulted.
>
> A is a sparse matrix that is created from a MEX function. I've experienced MATLAB segfaulting when performing operations such as sparse(A) or A' (transpose) when I fill the index wrongly within the MEXfunction. However, when MATLAB segfaults by calling A\b, I check prior to the A\b call if A was filled incorrectly by calling A' and sparse(A) and it passes just fine.
>
> A\b call can segfault after 10 calls, or after 1000 calls. There doesn't seem to be any pattern and I have no idea what's causing it.
>
> Thanks.
>
> -Will

First thing I would do is run SPOK on it to make sure you are building the sparse matrix correctly:

http://www.mathworks.com/matlabcentral/fileexchange/20186-spok-checks-if-a-matlab-sparse-matrix-is-ok

James Tursa

Hi James,

First of all, thanks for the amazing checking function. I checked it and I had unnecessary zeros in my sparse matrix so I was able to eliminate them. However, my raw assembly of sparse matrix seems to be fine.

Another suggestion I received from a colleague of mine is that, "ALWAYS" recompile and re-build mex functions whenever moving from one operating system and/or architecture to another. I did compile my mex function for linux operating system but moved my code to another linux operating system but a different architecture and didn't bother to recompile everything. Now, I recompiled everything and am re-running the simulation. I will post again if my simulation runs consistently for multiple runs.

Thanks.

-Will

"William Taitano" <williamtaitano1208@hotmail.com> wrote in message <k4nk01$jn8$1@newscl01ah.mathworks.com>...
>
> First of all, thanks for the amazing checking function. I checked it and I had unnecessary zeros in my sparse matrix so I was able to eliminate them. However, my raw assembly of sparse matrix seems to be fine.

If you need fast in-place mex sparse 0 removal functions you can download my MTIMESX submission on the FEX and look for the functions spcleanreal and spcleancomplex in the file mtimesx.c. You can find MTIMESX here:

http://www.mathworks.com/matlabcentral/fileexchange/25977-mtimesx-fast-matrix-multiply-with-multi-dimensional-support

The functions are pretty much black boxes since I started with something readable and then highly optimized them for speed ... which rendered the final code somewhat ugly ... well, quite a bit ugly actually. But they *are* pretty fast IMO.

James Tursa