contrib.bayesflow.entropy.entropy_shannon

tf.contrib.bayesflow.entropy.entropy_shannon

tf.contrib.bayesflow.entropy.entropy_shannon

entropy_shannon(
    p,
    z=None,
    n=None,
    seed=None,
    form=None,
    name='entropy_shannon'
)

Defined in tensorflow/contrib/bayesflow/python/ops/entropy_impl.py.

See the guide: BayesFlow Entropy (contrib) > Ops

Monte Carlo or deterministic computation of Shannon's entropy.

Depending on the kwarg form, this Op returns either the analytic entropy of the distribution p, or the sampled entropy:

-n^{-1} sum_{i=1}^n p.log_prob(z_i),  where z_i ~ p,
    \approx - E_p[ Log[p(Z)] ]
    = Entropy[p]

User supplies either Tensor of samples z, or number of samples to draw n

Args:

• p: tf.contrib.distributions.Distribution
• z: Tensor of samples from p, produced by p.sample(n) for some n.
• n: Integer Tensor. Number of samples to generate if z is not provided.
• seed: Python integer to seed the random number generator.
• form: Either ELBOForms.analytic_entropy (use formula for entropy of q) or ELBOForms.sample (sample estimate of entropy), or ELBOForms.default (attempt analytic entropy, fallback on sample). Default value is ELBOForms.default.
• name: A name to give this Op.

Returns:

A Tensor with same dtype as p, and shape equal to p.batch_shape.

Raises:

• ValueError: If form not handled by this function.
• ValueError: If form is ELBOForms.analytic_entropy and n was provided.