Fast indexing - more than linear indexing
6 views (last 30 days)
Show older comments
Hello all I am facing another problem. I have to manage with large arrays. I have a first array which length is 1 billion elements. Since the most part of elements are null, this array has been set as a sparse array. Inside of it, I selected 1 million of odd indices by means of randperm function. Now I have to place random elements belonging to [a, b] range inside of those 1 million elements. Here is a simple code
large_array = sparse(1e9,1);
inds_array = randperm(odd_indices, 1e6); % odd_indices contain odd indices of large_array
rand_array = a + (b-a)*rand(1e6, 1);
large_array(inds_array) = rand_array;
Now, I am experiencing that, for such a kind of situation where large number of elements are involved, the linear indexing is too slow anyway. Do you have any suggestion to speed up the procedure, please?
Thanks Matteo
2 Comments
the cyclist
on 23 Jun 2012
I do not understand the comment on the second line, about odd_indices. That input can't be a vector, right?
Is it possible for you to post complete code that we would be able to execute, instead of us guessing what you mean by odd_indices, a, and b?
Accepted Answer
Walter Roberson
on 24 Jun 2012
If the indexing is slow, do fewer indexing operations:
tinds = [inds(1:ind_temp), inds(num_half_indices+1:len_pos-2*mod(len_pos,2))];
tvals = [temp_rand, mod(temp_rand + pi, 2*pi)];
thetas(tinds) = tvals;
And if the two index ranges can overlap, then work that out so that you only do the unique indices.
Once you have it down to a single assignment call, combine creation of the sparse matrix with setting the values
thetas = sparse(tinds, 1, tvals, num_div^3, 1);
I have read that this is much faster than setting the values after the matrix is created.
More Answers (1)
per isakson
on 23 Jun 2012
Based on the use in RANDPERM, I assume odd_indices is a scalar integer.
PROFILE indicates that "all" time is spent in the last statement. I would say that the performance problem is with SPARSE rather than with linear indexing. The value of "1e6" is critical to the performance.
If memory allows the performance with a full matrix is much better.
--- Cont. ---
Have you analyzed the code with PROFILE?
Yes, it is the assignments of the sparse vector, thetas, that takes all the time.
How much faster execution do you need?
You say that the improvement, which is possible with a full matrix is not enough. I don't think it is possible achieve the performance you need by improvements on this code. One need to use another approach or another language.
See Also
Categories
Find more on Logical in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!