Relativistic GAN

A Relativistic GAN is a type of generative adversarial network. It has a relativistic discriminator which estimates the probability that the given real data is more realistic than a randomly sampled fake data. The idea is to endow GANs with the property that the probability of real data being real ($D\left(x\_{r}\right)$) should decrease as the probability of fake data being real ($D\left(x\_{f}\right)$) increases.

With a standard GAN, we can achieve this as follows. The standard GAN discriminator can be defined, in term of the non-transformed layer $C\left(x\right)$, as $D\left(x\right) = \text{sigmoid}\left(C\left(x\right)\right)$. A simple way to make discriminator relativistic - having the output of $D$ depend on both real and fake data - is to sample from real/fake data pairs $\tilde{x} = \left(x\_{r}, x\_{f}\right)$ and define it as $D\left(\tilde{x}\right) = \text{sigmoid}\left(C\left(x\_{r}\right) − C\left(x\_{f}\right)\right)$. The modification can be interpreted as: the discriminator estimates the probability that the given real data is more realistic than a randomly sampled fake data.

More generally a Relativistic GAN can be interpreted as having a discriminator of the form $a\left(C\left(x\_{r}\right)−C\left(x\_{f}\right)\right)$, where $a$ is the activation function, to be relativistic.