ICE: Removing duplicates from arrays

Here’s an ICE tree that removes all duplicate elements from an array. It uses Generate Sample Set, so there’s no repeat nodes. But it relies on the fact that you can feed in an array of indices into Remove from Array, and Remove from Array doesn’t care if that array of indices itself contains duplicate. So, if you plug the array [1,1,1,2,2,2,3,3,3] in the Index port, Remove from Array will nicely remove elements 1, 2, and 3 with no complaints.


Unlike some other methods, this works with scalars too:


Using Generate Sample Set to avoid the Repeat node

Hat tip to Oleg who posted this tip on the mailing list some time ago.

The basic ideas is that instead of looping over an array with a Repeat with Counter, you use Generate Sample Set to get a data set, and you do everything in a per-generated element context.


As an example, let’s revisit the problem of taking one array and creating a new array where each element is the sum of the elements before it.


The old way, with a Repeat with Counter node, looks like this:

Using Generate Sample Set, you can work with a data set and use that to access the array elements:

Generate Sample Set is set up to give an exact number of samples: