contrib.rnn.DropoutWrapper
tf.contrib.rnn.DropoutWrapper
class tf.contrib.rnn.DropoutWrapper
class tf.nn.rnn_cell.DropoutWrapper
Defined in tensorflow/python/ops/rnn_cell_impl.py
.
See the guide: RNN and Cells (contrib) > Core RNN Cell wrappers (RNNCells that wrap other RNNCells)
Operator adding dropout to inputs and outputs of the given cell.
Properties
graph
losses
non_trainable_variables
non_trainable_weights
output_size
scope_name
state_size
trainable_variables
trainable_weights
updates
variables
Returns the list of all layer variables/weights.
Returns:
A list of variables.
weights
Returns the list of all layer variables/weights.
Returns:
A list of variables.
Methods
__init__
__init__( cell, input_keep_prob=1.0, output_keep_prob=1.0, state_keep_prob=1.0, variational_recurrent=False, input_size=None, dtype=None, seed=None )
Create a cell with added input, state, and/or output dropout.
If variational_recurrent
is set to True
(NOT the default behavior), then the the same dropout mask is applied at every step, as described in:
Y. Gal, Z Ghahramani. "A Theoretically Grounded Application of Dropout in Recurrent Neural Networks". https://arxiv.org/abs/1512.05287
Otherwise a different dropout mask is applied at every time step.
Args:
-
cell
: an RNNCell, a projection to output_size is added to it. -
input_keep_prob
: unit Tensor or float between 0 and 1, input keep probability; if it is constant and 1, no input dropout will be added. -
output_keep_prob
: unit Tensor or float between 0 and 1, output keep probability; if it is constant and 1, no output dropout will be added. -
state_keep_prob
: unit Tensor or float between 0 and 1, output keep probability; if it is constant and 1, no output dropout will be added. State dropout is performed on the output states of the cell. -
variational_recurrent
: Python bool. IfTrue
, then the same dropout pattern is applied across all time steps per run call. If this parameter is set,input_size
must be provided. -
input_size
: (optional) (possibly nested tuple of)TensorShape
objects containing the depth(s) of the input tensors expected to be passed in to theDropoutWrapper
. Required and used iffvariational_recurrent = True
andinput_keep_prob < 1
. -
dtype
: (optional) Thedtype
of the input, state, and output tensors. Required and used iffvariational_recurrent = True
. -
seed
: (optional) integer, the randomness seed.
Raises:
-
TypeError
: if cell is not an RNNCell. -
ValueError
: if any of the keep_probs are not between 0 and 1.
__call__
__call__( inputs, state, scope=None )
Run the cell with the declared dropouts.
__deepcopy__
__deepcopy__(memo)
add_loss
add_loss( losses, inputs=None )
Add loss tensor(s), potentially dependent on layer inputs.
Some losses (for instance, activity regularization losses) may be dependent on the inputs passed when calling a layer. Hence, when reusing a same layer on different inputs a
and b
, some entries in layer.losses
may be dependent on a
and some on b
. This method automatically keeps track of dependencies.
The get_losses_for
method allows to retrieve the losses relevant to a specific set of inputs.
Arguments:
-
losses
: Loss tensor, or list/tuple of tensors. -
inputs
: Optional input tensor(s) that the loss(es) depend on. Must match theinputs
argument passed to the__call__
method at the time the losses are created. IfNone
is passed, the losses are assumed to be unconditional, and will apply across all dataflows of the layer (e.g. weight regularization losses).
add_update
add_update( updates, inputs=None )
Add update op(s), potentially dependent on layer inputs.
Weight updates (for instance, the updates of the moving mean and variance in a BatchNormalization layer) may be dependent on the inputs passed when calling a layer. Hence, when reusing a same layer on different inputs a
and b
, some entries in layer.updates
may be dependent on a
and some on b
. This method automatically keeps track of dependencies.
The get_updates_for
method allows to retrieve the updates relevant to a specific set of inputs.
Arguments:
-
updates
: Update op, or list/tuple of update ops. -
inputs
: Optional input tensor(s) that the update(s) depend on. Must match theinputs
argument passed to the__call__
method at the time the updates are created. IfNone
is passed, the updates are assumed to be unconditional, and will apply across all dataflows of the layer.
add_variable
add_variable( name, shape, dtype=None, initializer=None, regularizer=None, trainable=True )
Adds a new variable to the layer, or gets an existing one; returns it.
Arguments:
-
name
: variable name. -
shape
: variable shape. -
dtype
: The type of the variable. Defaults toself.dtype
. -
initializer
: initializer instance (callable). -
regularizer
: regularizer instance (callable). -
trainable
: whether the variable should be part of the layer's "trainable_variables" (e.g. variables, biases) or "non_trainable_variables" (e.g. BatchNorm mean, stddev).
Returns:
The created variable.
apply
apply( inputs, *args, **kwargs )
Apply the layer on a input.
This simply wraps self.__call__
.
Arguments:
-
inputs
: Input tensor(s). args: additional positional arguments to be passed toself.call
.
*kwargs: additional keyword arguments to be passed toself.call
.
Returns:
Output tensor(s).
build
build(_)
call
call( inputs, **kwargs )
The logic of the layer lives here.
Arguments:
-
inputs
: input tensor(s). **kwargs: additional keyword arguments.
Returns:
Output tensor(s).
get_losses_for
get_losses_for(inputs)
Retrieves losses relevant to a specific set of inputs.
Arguments:
-
inputs
: Input tensor or list/tuple of input tensors. Must match theinputs
argument passed to the__call__
method at the time the losses were created. If you passinputs=None
, unconditional losses are returned, such as weight regularization losses.
Returns:
List of loss tensors of the layer that depend on inputs
.
get_updates_for
get_updates_for(inputs)
Retrieves updates relevant to a specific set of inputs.
Arguments:
-
inputs
: Input tensor or list/tuple of input tensors. Must match theinputs
argument passed to the__call__
method at the time the updates were created. If you passinputs=None
, unconditional updates are returned.
Returns:
List of update ops of the layer that depend on inputs
.
zero_state
zero_state( batch_size, dtype )
© 2017 The TensorFlow Authors. All rights reserved.
Licensed under the Creative Commons Attribution License 3.0.
Code samples licensed under the Apache 2.0 License.
https://www.tensorflow.org/api_docs/python/tf/contrib/rnn/DropoutWrapper