This Deduplicate Array 1.2 compound from Julian Johnson can remove duplicate values from any type of array: integer, scalar, location, and vector.

Deduplicate Array uses Find in Array, which can find all occurrences of a value in the array. That means it pushes some of the looping down to a compiled FindInArray node, and gives a little better performance than if you did all the looping in ICE with Repeat nodes. A fully compiled node would probably be even faster (see this XSI list post).

To understand/visualize how Deduplicate works, I took the body of its Repeat loop, disconnected it, duplicated it six times, and plugged them all into an Execute node. That way I could Show Values for each “iteration” of the loop. So, for example, with my test array of 14 elements, I could see that took just 4 loops to remove all the duplicates. After that, in the real Deduplicate compound, each loop would basically do nothing (so I suppose you could convert it to a While loop?)

To compare performance, I built a simple “remove duplicates” tree that didn’t use Find in Array (I just used one of the first algorithms I found on google):

In one test, Deduplicate took 3 seconds while my ICE tree took 14!