contrib.lookup.index_table_from_tensor

tf.contrib.lookup.index_table_from_tensor

tf.contrib.lookup.index_table_from_tensor

index_table_from_tensor(
    mapping,
    num_oov_buckets=0,
    default_value=-1,
    hasher_spec=tf.contrib.lookup.FastHashSpec,
    dtype=tf.string,
    name=None
)

Defined in tensorflow/contrib/lookup/lookup_ops.py.

Returns a lookup table that converts a string tensor into int64 IDs.

This operation constructs a lookup table to convert tensor of strings into int64 IDs. The mapping can be initialized from a string mapping 1-D tensor where each element is a key and corresponding index within the tensor is the value.

Any lookup of an out-of-vocabulary token will return a bucket ID based on its hash if num_oov_buckets is greater than zero. Otherwise it is assigned the default_value. The bucket ID range is [mapping size, mapping size + num_oov_buckets].

The underlying table must be initialized by calling tf.tables_initializer.run() or table.init.run() once.

Elements in mapping cannot have duplicates, otherwise when executing the table initializer op, it will throw a FailedPreconditionError.

Sample Usages:

mapping_strings = t.constant(["emerson", "lake", "palmer"])
table = tf.contrib.lookup.index_table_from_tensor(
    mapping=mapping_strings, num_oov_buckets=1, default_value=-1)
features = tf.constant(["emerson", "lake", "and", "palmer"])
ids = table.lookup(features)
...
tf.tables_initializer().run()

ids.eval()  ==> [0, 1, 4, 2]

Args:

• mapping: A 1-D Tensor that specifies the mapping of keys to indices. The type of this object must be castable to dtype.
• num_oov_buckets: The number of out-of-vocabulary buckets.
• default_value: The value to use for out-of-vocabulary feature values. Defaults to -1.
• hasher_spec: A HasherSpec to specify the hash function to use for assignment of out-of-vocabulary buckets.
• dtype: The type of values passed to lookup. Only string and integers are supported.
• name: A name for this op (optional).

Returns:

The lookup table to map an input Tensor to index int64 Tensor.

Raises:

• ValueError: If mapping is invalid.
• ValueError: If num_oov_buckets is negative.