江苏龙网

  1. 首页 > 生活 > >

怎么样在tensorflow中使用batch normalization

在高票（作者：于洋，链接：于洋：怎样在tensorflow中使用batch normalization？）基础的细节更改。主要减少了不比要的moments计算。参数仿造pytorch
from tensorflow.python.training.moving_averages import assign_moving_averagedef batch_norm(x, train, eps=1e-05, decay=0.9, affine=True, name=None): with tf.variable_scope(name, default_name=\u0026#39;BatchNorm2d\u0026#39;): params_shape = tf.shape(x) moving_mean = tf.get_variable(\u0026#39;mean\u0026#39;, params_shape, initializer=tf.zeros_initializer, trainable=False) moving_variance = tf.get_variable(\u0026#39;variance\u0026#39;, params_shape, initializer=tf.ones_initializer, trainable=False) def mean_var_with_update(): mean, variance = tf.nn.moments(x, tf.shape(x), name=\u0026#39;moments\u0026#39;) with tf.control_dependencies(): return tf.identity(mean), tf.identity(variance) mean, variance = tf.cond(train, mean_var_with_update, lambda: (moving_mean, moving_variance)) if affine: beta = tf.get_variable(\u0026#39;beta\u0026#39;, params_shape, initializer=tf.zeros_initializer) gamma = tf.get_variable(\u0026#39;gamma\u0026#39;, params_shape, initializer=tf.ones_initializer) x = tf.nn.batch_normalization(x, mean, variance, beta, gamma, eps) else: x = tf.nn.batch_normalization(x, mean, variance, None, None, eps) return x
■网友
首先，version 1有问题。
其次，contrib.layers和slim里的bn是完全一样的。看看slim的__init__。
再次，于洋的train_phase是是否在training的tag。
最后，slim的bn肯定没问题，但不是直接调用下就完事了，你若只调用下，会发现acc上下起伏，连绵不绝。。。moving_mean 和 moving_variance是需要更新的，利用control_dependencies保证每次训练前更新一下。所以思路上我觉得于洋的回答可用。若要直接调用slim的bn，你的update_op要和train_op保证一个dependency。
最最后，多看tf的官方文档，bn怎么用，写的还是很清楚的。

■网友

What is right batch normalization function in Tensorflow?这个Stack Overflow回答详解了目前tensorflow中所有的batch normalization用法，其中推荐使用的high-level API是tf.layers.batch_normalization。如果想用low-level API自己写函数，则用tf.nn.batch_normalization即可。

■网友
示例代码见Udacity的batch-norm教程，你会找到三个 notebook：
Batch_Normalization_Lesson.ipynb - 此 notebook 将向你展示批归一化的原理Batch_Normalization_Exercises.ipynb - 由你来实现批归一化的练习Batch_Normalization_Solutions.ipynb - 这些练习的解决方案Tips：
Added is_training, a placeholder to store a boolean value indicating whether or not the network is training.Passed is_training to the conv_layer and fully_connected functions.Each time we call run on the session, we added to feed_dict the appropriate value for is_training.Moved the creation of train_opt inside a with tf.control_dependencies... statement. This is necessary to get the normalization layers created with tf.layers.batch_normalization to update their population statistics, which we need when performing inference.


推荐阅读


上一篇：准研究生一枚 给一名教授发邮件说想成为她的学生 但是老师昨晚回复：内容太简单 没懂啊 该咋回复呢

下一篇：小米MIX的光线自动调节功能还有吗