I recently started to study non-parametric clustering methods and I come across to CRP. After reading all the material I found on the web there is one thing which is not completely clear to me: which is the aim of this process? You can't cluster according to 'distance' (and in fact it was invented another method called Distance Dependent CRP), so why should someone apply this technique? Being a 'noob' I am sure I am missing something... What?
The Chinese Restaurant Process is a way of looking at a Dirichlet process. It is a distribution over distributions. There are various ways of thinking about either. One way of looking at it is that as you draw samples, for each new sample:
- there is a finite probability that the new sample is assigned to an existing cluster
- and otherwise it becomes the first sample in a new cluster
So, you don't have to specify how many clusters there are, which is why it's 'non-parametric'. You do have to specify the probability of each new sample being assigned to a new cluster however, so there is a parameter for that, eg
Then, you use the DP ("Dirichlet Process") as part of a model, feed it data, and use some way to solve the model. Typically, the model is analytically non-tractable, so one can use Markov Chain Monte Carlo or Variational Approximation to solve it, and get some estimation of the number of clusters, given the data, and the
Note that I've simplified a little: the DP is a distribution over distributions. The samples drawn above correspond to samples from a single draw from the distribution over distributions. One can repeat the process of drawing samples, in order to draw samples from a new draw from the distribution over distributions...