admin管理员组文章数量:1579945
Keras 中有leaky_relu的实现。leaky_relu被整合进了relu函数。
参考官方文档:
https://tensorflow.google/api_docs/python/tf/keras/backend/relu?hl=en
Arguments | |
---|---|
x | A tensor or variable. |
alpha | A scalar, slope of negative section (default=0.). |
max_value | float. Saturation threshold. |
threshold | float. Threshold value for thresholded activation. |
alpha(超参数)值控制负数部分线性函数的梯度。当alpha = 0 ,是原始的relu函数。当alpha >0,即为leaky_relu。
查看源码,在Keras.backbend 中,也是调用tensorflow.python.ops库nn中的leaky_relu函数实现的:
def relu(x, alpha=0., max_value=None, threshold=0):
"""Rectified linear unit.
With default values, it returns element-wise `max(x, 0)`.
Otherwise, it follows:
`f(x) = max_value` for `x >= max_value`,
`f(x) = x` for `threshold <= x < max_value`,
`f(x) = alpha * (x - threshold)` otherwise.
Arguments:
x: A tensor or variable.
alpha: A scalar, slope of negative section (default=`0.`).
max_value: float. Saturation threshold.
threshold: float. Threshold value for thresholded activation.
Returns:
A tensor.
"""
if alpha != 0.:
if max_value is None and threshold == 0:
return nn.leaky_relu(x, alpha=alpha) ##在这里调用了leaky_relu
if threshold != 0:
negative_part = nn.relu(-x + threshold)
else:
negative_part = nn.relu(-x)
clip_max = max_value is not None
if threshold != 0:
# computes x for x > threshold else 0
x = x * math_ops.cast(math_ops.greater(x, threshold), floatx())
elif max_value == 6:
# if no threshold, then can use nn.relu6 native TF op for performance
x = nn.relu6(x)
clip_max = False
else:
x = nn.relu(x)
if clip_max:
max_value = _constant_to_tensor(max_value, x.dtype.base_dtype)
zero = _constant_to_tensor(0, x.dtype.base_dtype)
x = clip_ops.clip_by_value(x, zero, max_value)
if alpha != 0.:
alpha = _to_tensor(alpha, x.dtype.base_dtype)
x -= alpha * negative_part
return x
版权声明:本文标题:Keras 调用leaky_relu 内容由热心网友自发贡献,该文观点仅代表作者本人, 转载请联系作者并注明出处:https://m.elefans.com/dongtai/1727845057a1132990.html, 本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如发现本站有涉嫌抄袭侵权/违法违规的内容,一经查实,本站将立刻删除。
发表评论