Projection Discriminator

A Projection Discriminator is a type of discriminator for generative adversarial networks. It is motivated by a probabilistic model in which the distribution of the conditional variable $\textbf{y}$ given $\textbf{x}$ is discrete or uni-modal continuous distributions.

If we look at the original solution for the loss function $\mathcal{L}\_{D}$ in the vanilla GANs, we can decompose it into the sum of two log-likelihood ratios:

$f^{*}\left(\mathbf{x}, \mathbf{y}\right) = \log\frac{q\left(\mathbf{x}\mid{\mathbf{y}}\right)q\left(\mathbf{y}\right)}{p\left(\mathbf{x}\mid{\mathbf{y}}\right)p\left(\mathbf{y}\right)} = \log\frac{q\left(\mathbf{y}\mid{\mathbf{x}}\right)}{p\left(\mathbf{y}\mid{\mathbf{x}}\right)} + \log\frac{q\left(\mathbf{x}\right)}{p\left(\mathbf{x}\right)} = r\left(\mathbf{y\mid{x}}\right) + r\left(\mathbf{x}\right)$

We can model the log likelihood ratio $r\left(\mathbf{y\mid{x}}\right)$ and $r\left(\mathbf{x}\right)$ by some parametric functions $f\_{1}$ and $f\_{2}$ respectively. If we make a standing assumption that $p\left(y\mid{x}\right)$ and $q\left(y\mid{x}\right)$ are simple distributions like those that are Gaussian or discrete log linear on the feature space, then the parametrization of the following form becomes natural:

$f\left(\mathbf{x}, \mathbf{y}; \theta\right) = f\_{1}\left(\mathbf{x}, \mathbf{y}; \theta\right) + f\_{2}\left(\mathbf{x}; \theta\right) = \mathbf{y}^{T}V\phi\left(\mathbf{x}; \theta\_{\phi}\right) + \psi\left(\phi(\mathbf{x}; \theta\_{\phi}); \theta\_{\psi}\right)$

where $V$ is the embedding matrix of $y$, $\phi\left(·, \theta\_{\phi}\right)$ is a vector output function of $x$, and $\psi\left(·, \theta\_{\psi}\right)$ is a scalar function of the same $\phi\left(\mathbf{x}; \theta\_{\phi}\right)$ that appears in $f\_{1}$. The learned parameters $\theta =${$V, \theta\_{\phi}, \theta\_{\psi}$} are trained to optimize the adversarial loss. This model of the discriminator is the projection.