blob: c5004ce3a3f11799006a2ff5450d8b53c5d3091b (
plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
|
--[[
This file implements Batch Normalization as described in the paper:
"Batch Normalization: Accelerating Deep Network Training
by Reducing Internal Covariate Shift"
by Sergey Ioffe, Christian Szegedy
This implementation is useful for inputs coming from convolution layers.
For non-convolutional layers, see BatchNormalization.lua
The operation implemented is:
y = ( x - mean(x) )
-------------------- * gamma + beta
standard-deviation(x)
where gamma and beta are learnable parameters.
The learning of gamma and beta is optional.
Usage:
with learnable parameters: nn.SpatialBatchNormalization(N [,eps] [,momentum])
where N = dimensionality of input
without learnable parameters: nn.SpatialBatchNormalization(N [,eps] [,momentum], false)
eps is a small value added to the variance to avoid divide-by-zero.
Defaults to 1e-5
In training time, this layer keeps a running estimate of it's computed mean and std.
The running sum is kept with a default momentum of 0.1 (unless over-ridden)
In test time, this running mean/std is used to normalize.
]]--
local BN, parent = torch.class('nn.SpatialBatchNormalization', 'nn.BatchNormalization')
BN.__version = 2
-- expected dimension of input
BN.nDim = 4
|
