Projection, Projection, Projection

[rweba]

So I was a bit curious as to how projections other than PCA might work on the senate data.

Random Projection: So this looks pretty easy, just multiply by a random matrix! But how could that possibly work? In fact it sounds stupid! Well according to some complicated math that I don't really understand: it could actually work pretty well:

2007-

Comments

[jcreed]

I'm really enjoying these posts!

If a random projection tends to separate clusters better than you'd intuitively expect, does this mean the two-party system is more "evolutionarily likely" (only in the sense of evolution of political systems, not of humans) than we expect? Like, if you take a random voter, their random priorities are perhaps more likely to line up approximately comfortably with one party or the other.

[rweba]

Well I think the idea is that the random projection "preserves distances" more than you'd intuitively expect. So if the clusters were already separated in the high dimensional space then they'd still have some degree of separation in the lower dimensional space. But if there were not separated in the high dimension random projection is not going to separate them (as I understand it).

So in this case I think the data is already separated and thats why random projection can separate it pretty decently (but not as well as PCA or MDS).

So I don't think if you took a set of random vectors and then did random projection you would necessarily end up with two clusters.

[gustavolacerda]

right, but the number of clusters tends to be preserved.

[rweba]

Yes, the number of clusters should be (approximately) preserved.