ML Reviews

bayes-rule

Canonically, for events xx and yy, we have the definition:

p(xy)=p(yx)p(x)p(y)p(x \vert y) = \frac{p(y \vert x)p(x)}{p(y)}

where p(y)p(y) is the [[evidence]], p(x)p(x) is the [[prior]], p(xy)p(x \vert y) is the [[posterior distribution]], and p(yx)p(y \vert x) is the [[conditional-distribution]] known as the likelihood.

Another way written, as seen in Josh Speagle's slides, is to write it as:

Pr(ΘD,M)=Pr(DΘ,M)Pr(ΘM)Pr(DM)\text{Pr}(\Theta \vert D,M) = \frac{\text{Pr}(D \vert \Theta, M)\text{Pr}(\Theta \vert M)}{\text{Pr}(D \vert M)}

where aspects are more clearly defined as MM model dependent (such as the prior), for model parameters Θ\Theta and data DD.