class MXNet::Optimizer::SGD

Overview

The SGD optimizer with momentum and weight decay.

Updates are calculated by:

rescaled_grad = lr * (rescale_grad * clip(grad, clip_gradient) + wd * weight)
state = momentum * state + rescaled_grad
weight = weight - state

Defined in:

mxnet/optimizer.cr

Constructors

Instance Method Summary

Instance methods inherited from class MXNet::Optimizer

create_state(index, weight) create_state, lr : Float64 lr, rescale_grad : Float64 rescale_grad, rescale_grad=(rescale_grad) rescale_grad=, set_lr_mult(lr_mult) set_lr_mult, set_wd_mult(wd_mult) set_wd_mult, update(index, weight, gradient, state) update, wd : Float64 wd

Constructor methods inherited from class MXNet::Optimizer

new(rescale_grad = 1.0, clip_gradient = -1.0, lr = 0.01, wd = 0.0) new

Class methods inherited from class MXNet::Optimizer

create(optimizer, **kwargs) create

Constructor Detail

def self.new(momentum = 0.0, **kwargs) #

Creates a new instance.

This optimizer accepts the following parameters in addition to those accepted by Optimizer.

Parameters

  • momentum (Float, optional) The momentum value.

[View source]

Instance Method Detail

def create_state(index, weight) #

[View source]
def update(index, weight, gradient, state) #

[View source]