My experience with applied statistics has been fun in large part because it involved making the Bayesian rubber meet the road of inherent computational complexity, and because it allowed me to define and explore theoretical concepts and tie them to our applied problem... I also had a good time formalizing and unifying ideas, making them more ( Read more... )

( Read more... )

Are likelihood functions ever non-differentiable?

Of course, you can always transform the parameters in such a way that you get a kink in the function... My question is whether this ever occurs naturally.

---

UPDATE: clearly, if the likelihood for each data point is differentiable, then so is the likelihood for the whole data.

I'm interested in the effective sample size of my Gibbs sample.

Plotting the autocorrelation function, I notice that the even lags have a higher much autocorrelation than the odd lags. But if you skip odd numbers (or the even numbers), it looks smooth. It's interesting to consider what may have caused this.

I'm also wondering if R's effectiveSize ( Read more... )

I am wondering if, instead of running MCMC and hoping that it has "mixed" ("achieved stationarity"), there are approaches based on computing (or approximating) the principal left eigenvector of the transition matrix ( Read more... )

This is paper draft that is probably more worthy of being a blog post, so I decided to blogize it. Here we go:

Abstract:
It is well-known that Independent Component Analysis (ICA) is not identifiable when more than one source is Gaussian (Comon 1993). However, the question of how identifiable it is is rarely asked. In this paper, we show a ( Read more... )

Being fond of non-parametrics, I have an instinctive dislike of Gaussian models (maybe because they are overused by people who don't know statistics).

Q: So why might one be satisfied with the Gaussian model, while knowing that it is very wrong (e.g. if you know the truth should be a skewed, fat-tailed distribution, e.g. a power-law distributionA ( Read more... )

Can one estimate the correlation between two variables when no joint data is observed?

Imagine you have samples from a 2D joint distribution... except that you don't have joint observations, because the x and y observations are unmatched. Your data can be visualized using vertical and horizontal lines (red lines on figure below).


( Read more... )

According to the sources I've read, Gibbs sampling doesn't really tell you how to sample... it's just divide-and-conquer: you still need to sample each variable from its full conditional ( Read more... )

What do you call a bootstrap in which you sample without replacement? This way the resamples will tend to look like plausible samples from the population, albeit smaller than the original sample.

Besides, you don't want to sample with replacement if you're:
(a) working with Chinese Restaurant Processes: you could be falsely led to believe that ( Read more... )