contrib.layers.variance_scaling_initializer
tf.contrib.layers.variance_scaling_initializer
tf.contrib.layers.variance_scaling_initializer
variance_scaling_initializer( factor=2.0, mode='FAN_IN', uniform=False, seed=None, dtype=tf.float32 )
Defined in tensorflow/contrib/layers/python/layers/initializers.py
.
See the guide: Layers (contrib) > Initializers
Returns an initializer that generates tensors without scaling variance.
When initializing a deep network, it is in principle advantageous to keep the scale of the input variance constant, so it does not explode or diminish by reaching the final layer. This initializer use the following formula:
if mode='FAN_IN': # Count only number of input connections. n = fan_in elif mode='FAN_OUT': # Count only number of output connections. n = fan_out elif mode='FAN_AVG': # Average number of inputs and output connections. n = (fan_in + fan_out)/2.0 truncated_normal(shape, 0.0, stddev=sqrt(factor / n))
- To get Delving Deep into Rectifiers, use (Default):
factor=2.0 mode='FAN_IN' uniform=False
- To get Convolutional Architecture for Fast Feature Embedding, use:
factor=1.0 mode='FAN_IN' uniform=True
- To get Understanding the difficulty of training deep feedforward neural networks, use:
factor=1.0 mode='FAN_AVG' uniform=True.
- To get
xavier_initializer
use either:
factor=1.0 mode='FAN_AVG' uniform=True
, or
factor=1.0 mode='FAN_AVG' uniform=False
.
Args:
-
factor
: Float. A multiplicative factor. -
mode
: String. 'FAN_IN', 'FAN_OUT', 'FAN_AVG'. -
uniform
: Whether to use uniform or normal distributed random initialization. -
seed
: A Python integer. Used to create random seeds. Seetf.set_random_seed
for behavior. -
dtype
: The data type. Only floating point types are supported.
Returns:
An initializer that generates tensors with unit variance.
Raises:
-
ValueError
: ifdtype
is not a floating point type. -
TypeError
: ifmode
is not in ['FAN_IN', 'FAN_OUT', 'FAN_AVG'].
© 2017 The TensorFlow Authors. All rights reserved.
Licensed under the Creative Commons Attribution License 3.0.
Code samples licensed under the Apache 2.0 License.
https://www.tensorflow.org/api_docs/python/tf/contrib/layers/variance_scaling_initializer