Got Questions? Get Answers.
Discover MakerZone

MATLAB and Simulink resources for Arduino, LEGO, and Raspberry Pi

Learn more

Discover what MATLAB® can do for your career.

Opportunities for recent engineering grads.

Apply Today

Thread Subject:
Parallelized sampling from a multivariate normal distribution

Subject: Parallelized sampling from a multivariate normal distribution

From: Misha Koshelev

Date: 14 Aug, 2009 02:07:01

Message: 1 of 4

Dear Sirs:

We are currently using a Gibbs Sampler with Markov Chain Monte Carlo to cluster individuals based on behavioral and brain imaging (functional MRI) data. We would like to expand our algorithm to include a much larger dataset - the main bottleneck is drawing from a multivariate normal distribution with quite a few variables.

We are interested in parallelizing sampling from such a distribution - and were wondering if this had been done/was possible, whether in MATLAB or otherwise.

Your replies very much appreciated.

Sincerely yours
Misha Koshelev

Subject: Parallelized sampling from a multivariate normal distribution

From: Peter Perkins

Date: 14 Aug, 2009 12:38:36

Message: 2 of 4

Misha Koshelev wrote:

> We are currently using a Gibbs Sampler with Markov Chain Monte Carlo to cluster individuals based on behavioral and brain imaging (functional MRI) data. We would like to expand our algorithm to include a much larger dataset - the main bottleneck is drawing from a multivariate normal distribution with quite a few variables.

Misha, it seems unlikely that the actual generation is the bottleneck. Perhaps you mean the factoring of the MVN covariance matrix?


> We are interested in parallelizing sampling from such a distribution - and were wondering if this had been done/was possible, whether in MATLAB or otherwise.

Are you looking to run parallel MCMC chains, or to generate a single very high-dimensional MVN vector using parallel computations?

Subject: Parallelized sampling from a multivariate normal distribution

From: Misha Koshelev

Date: 14 Aug, 2009 16:52:20

Message: 3 of 4

Peter Perkins <Peter.Perkins@MathRemoveThisWorks.com> wrote in message <h63lsc$pqg$1@fred.mathworks.com>...
> Misha Koshelev wrote:
>
> > We are currently using a Gibbs Sampler with Markov Chain Monte Carlo to cluster individuals based on behavioral and brain imaging (functional MRI) data. We would like to expand our algorithm to include a much larger dataset - the main bottleneck is drawing from a multivariate normal distribution with quite a few variables.
>
> Misha, it seems unlikely that the actual generation is the bottleneck. Perhaps you mean the factoring of the MVN covariance matrix?
Yes. That is exactly what I mean. Is there a good parallel algorithm for this by any chance?

>
>
> > We are interested in parallelizing sampling from such a distribution - and were wondering if this had been done/was possible, whether in MATLAB or otherwise.
>
> Are you looking to run parallel MCMC chains, or to generate a single very high-dimensional MVN vector using parallel computations?

We would in fact like to generate a very high-dimensional MVN vector using parallel computations (perhaps 60,000 independent variables... or more).

Any advice much appreciated.

Thank you
Misha

Subject: Parallelized sampling from a multivariate normal distribution

From: Peter Perkins

Date: 14 Aug, 2009 19:01:24

Message: 4 of 4

Misha Koshelev wrote:
> Peter Perkins <Peter.Perkins@MathRemoveThisWorks.com> wrote in message <h63lsc$pqg$1@fred.mathworks.com>...
>> Misha Koshelev wrote:
>>
>>> We are currently using a Gibbs Sampler with Markov Chain Monte Carlo to cluster individuals based on behavioral and brain imaging (functional MRI) data. We would like to expand our algorithm to include a much larger dataset - the main bottleneck is drawing from a multivariate normal distribution with quite a few variables.
>> Misha, it seems unlikely that the actual generation is the bottleneck. Perhaps you mean the factoring of the MVN covariance matrix?
> Yes. That is exactly what I mean. Is there a good parallel algorithm for this by any chance?

Misha, I can't tell if you're using MATLAB or not. The CHOL function in MATLAB is multithreaded to some extent, so if you run on a multicore machine, you may see some speed-up. You may be able to use a distributed version of CHOL with the Parallel Computing Toolbox. It's hard to say from the information you've provided, and likely only trying it will tell you if it is faster.


>>> We are interested in parallelizing sampling from such a distribution - and were wondering if this had been done/was possible, whether in MATLAB or otherwise.
>> Are you looking to run parallel MCMC chains, or to generate a single very high-dimensional MVN vector using parallel computations?
>
> We would in fact like to generate a very high-dimensional MVN vector using parallel computations (perhaps 60,000 independent variables... or more).

Obviously that will be too big to fit in memory as a full matrix. If most of the covariances ar zero, you could use a sparse matrix. You can do that without multicore or PCT.

Tags for this Thread

No tags are associated with this thread.

What are tags?

A tag is like a keyword or category label associated with each thread. Tags make it easier for you to find threads of interest.

Anyone can tag a thread. Tags are public and visible to everyone.

Contact us