自建网站做电商,安卓小程序制作,太原快速排名,广州网站建设程序员培训MXNet的API mxnet里面的model API不是真的API#xff0c;它只不过是一个对ndarray的一个封装#xff0c;使其更容易使用。训练一个模型 为了训练一个模型#xff0c;你需要遵循以下两步#xff0c;第一步是使用symbol来构造#xff0c;然后调用model.Feedforward.create这…MXNet的API
mxnet里面的model API不是真的API它只不过是一个对ndarray的一个封装使其更容易使用。训练一个模型
为了训练一个模型你需要遵循以下两步第一步是使用symbol来构造然后调用model.Feedforward.create这个方法来创建一个model。下面的代码创建了一个两层的神经网络。# configure a two layer neuralnetwork
data mx.symbol.Variable(data)
fc1 mx.symbol.FullyConnected(data, namefc1, num_hidden128)
act1 mx.symbol.Activation(fc1, namerelu1, act_typerelu)
fc2 mx.symbol.FullyConnected(act1, namefc2, num_hidden64)
softmax mx.symbol.SoftmaxOutput(fc2, namesm)
# create a model
model mx.model.FeedForward.create(softmax,Xdata_set,num_epochnum_epoch,learning_rate0.01) 你还可以使用scikit-learn一样的风格来构造和拟合一个模型# create a model using sklearn-style two step way
model mx.model.FeedForward(softmax,num_epochnum_epoch,learning_rate0.01)model.fit(Xdata_set) 你如果想看更多的功能请看Model API Reference保存模型 # save a model to mymodel-symbol.json and mymodel-0100.params
prefix mymodel
iteration 100
model.save(prefix, iteration)# load model back
model_loaded mx.model.FeedForward.load(prefix, iteration) 我们往往用一个脚本进行对数据的训练往往以前缀加序号的形式如mymodel-0100.params这样的形式保存然后用另一个脚本加载模型并进行预测来完成相应的功能。阶段性的点检测(Checkpoint)
我们进行周期性的点检测是很有必要的。为了做这个你只要简单的加一个回调函数do_checkpoint(path)在函数里面。这个训练的过程将会自动的在每次迭代的时候在特殊的位置进行点检测。prefixmodels/chkpt
model mx.model.FeedForward.create(softmax,Xdata_set,iter_end_callbackmx.callback.do_checkpoint(prefix),...) 你可以加载模型的点检测在使用Feedforward.load之后。使用多个设备
简单的设置ctx其内容为你要训练设备(cpu,gpu)的列表。devices [mx.gpu(i) for i in range(num_device)]
model mx.model.FeedForward.create(softmax,Xdataset,ctxdevices,...) 这个训练过程将会通过一个并行的方式在你指定的GPUS进行。模型API
MXNet模型模块 mxnet.model.BatchEndParam¶ alias of BatchEndParams BatchEndParam是BatchEndParams的参数mxnet.model.save_checkpoint(prefix, epoch, symbol, arg_params, aux_params) Checkpoint the model data into file. Parameters: prefix (str) – Prefix of model name.epoch (int) – The epoch number of the model.symbol (Symbol) – The input symbolarg_params (dict of str to NDArray) – Model parameter, dict of name to NDArray of net’s weights.aux_params (dict of str to NDArray) – Model parameter, dict of name to NDArray of net’s auxiliary states. Notes prefix-symbol.json will be saved for symbol.prefix-epoch.params will be saved for parameters. 类功能对模型数据点检测后存入到文件中。参数prefix(str)-模型名的前缀(可以是个文件夹)epoch(int)-模型的epoch的数量(epoch在机器学习里面指的是把所有的样本进行一次全部操作(前向传播反向传播等等),和普通的迭代相比epoch的尺度比较大)symbol(Symbol)-输入的symbol。arg_params(一个NDArray的字符字典)-模型参数以及网络权重字典。aux_params(一个NDArray的字符字典)-模型参数以及一些附加状态的字典。Notes prefix-symbol.json will be saved for symbol.prefix-epoch.params will be saved for parameters. 注意:prefix-symbol.json将会存储symbol。prefix-epoch.params会存储参数。一个模型的symbol文件往往是唯一确定的而params文件可以很多最后你可以把一些没用的params文件给删掉。一般params的个数等于epoch的个数因为越往后面的params越好所以你可以只保留最后一个的params文件。mxnet.model.load_checkpoint(prefix, epoch) Load model checkpoint from file. Parameters: prefix (str) – Prefix of model name.epoch (int) – Epoch number of model we would like to load. Returns: symbol (Symbol) – The symbol configuration of computation network.arg_params (dict of str to NDArray) – Model parameter, dict of name to NDArray of net’s weights.aux_params (dict of str to NDArray) – Model parameter, dict of name to NDArray of net’s auxiliary states. 类功能:加载检测点(感觉还是翻译成检测点比较好)参数:prefix(str)-模型名称的前缀epoch(int)-你想加载的模型的epoch的序号一般是最大的那个。返回值:symbol(Symbol)-我们要计算网络的模型配置arg_params(一个NDArray的字符字典)-模型参数以及网络权重字典。aux_params(一个NDArray的字符字典)-模型参数以及一些附加状态的字典。class mxnet.model.FeedForward(symbol, ctxNone, num_epochNone, epoch_sizeNone,optimizersgd, initializermxnet.initializer.Uniform object, numpy_batch_size128,arg_paramsNone, aux_paramsNone, allow_extra_paramsFalse, begin_epoch0, **kwargs)¶ Model class of MXNet for training and predicting feedforward nets. This class is designed for a single-data single output supervised network. Parameters: symbol (Symbol) – The symbol configuration of computation network.ctx (Context or list of Context, optional) – The device context of training and prediction. To use multi GPU training, pass in a list of gpu contexts.num_epoch (int, optional) – Training parameter, number of training epochs(epochs).epoch_size (int, optional) – Number of batches in a epoch. In default, it is set to ceil(num_train_examples / batch_size)optimizer (str or Optimizer, optional) – Training parameter, name or optimizer object for training.initializer (initializer function, optional) – Training parameter, the initialization scheme used.numpy_batch_size (int, optional) – The batch size of training data. Only needed when input array is numpy.arg_params (dict of str to NDArray, optional) – Model parameter, dict of name to NDArray of net’s weights.aux_params (dict of str to NDArray, optional) – Model parameter, dict of name to NDArray of net’s auxiliary states.allow_extra_params (boolean, optional) – Whether allow extra parameters that are not needed by symbol to be passed by aux_params and arg_params. If this is True, no error will be thrown when aux_params and arg_params contain extra parameters than needed.begin_epoch (int, optional) – The begining training epoch.kwargs (dict) – The additional keyword arguments passed to optimizer. 类功能:MXNet的用来训练和预测前向传播网络的模型类。这个类设计来是为了得到一个单一输出的监督网络。参数:symbol(Symbol)-计算网络的symbol构造。ctx(Context or list of Context,optional)-用来训练和预测的设备。如果要使用多个GPU请传入gpu上下文。num_epoch(int,optional)-训练epoches的个数。epoch_size(int,optional)- 一个epoch里面的batch的个数。默认ceil(num_train_examples/batch_size)即训练的样本的个数/batch的大小然后取整。optimizer(str or Optimizer,optional)-训练参数名字或者相应的优化类用来训练的。initializer(initializer function,optional)-训练参数用来初始化的组合。numpy_batch_size(int,optional)-训练集的batch尺寸。只有当输入的数组是numpy的时候需要。arg_params(一个NDArray的字符字典)-模型参数以及网络权重字典。 aux_params(一个NDArray的字符字典)-模型参数以及一些附加状态的字典。 allow_extra_params(boolean,optional)-是否需要一些额外的参数aux_params和arg_params不需要的。如果这是真的那么就不会抛出错误当参数的个数超出所需要的参数的时候。begin_epoch(int,optional)-开始训练的epoch也就是说这一epoch后面的epoch都会重新训练。kwargs(dict)-额外的关键参数被传到optimizer里面的。predict(X, num_batchNone, return_dataFalse, resetTrue)¶ Run the prediction, always only use one device. :param X: :type X: mxnet.DataIter :param num_batch: the number of batch to run. Go though all batches if None :type num_batch: int or None Returns:y – The predicted value of the output.Return type:numpy.ndarray or a list of numpy.ndarray if the network has multiple outputs. 类方法功能:进行预测,只能使用一个device.参数X是X类型的batch的运行数量如果被设置为None的话会对里面的所有的批进行处理。返回值我们的预测值。score(X, eval_metricacc, num_batchNone, batch_end_callbackNone, resetTrue) Run the model on X and calculate the score with eval_metric :param X: :type X: mxnet.DataIter :param eval_metric: The metric for calculating score :type eval_metric: metric.metric :param num_batch: the number of batch to run. Go though all batches if None :type num_batch: int or None Returns:s – the final scoreReturn type:float 类方法功能:在X上运行模型并且用评估矩阵计算分数。返回值我们的最终分数。fit(X, yNone, eval_dataNone, eval_metricacc, epoch_end_callbackNone,batch_end_callbackNone, kvstorelocal, loggerNone, work_load_listNone, monitorNone,eval_batch_end_callbackNone) Fit the model. Parameters: X (DataIter, or numpy.ndarray/NDArray) – Training data. If X is an DataIter, the name or, if not available, position, of its outputs should match the corresponding variable names defined in the symbolic graph.y (numpy.ndarray/NDArray, optional) – Training set label. If X is numpy.ndarray/NDArray, y is required to be set. While y can be 1D or 2D (with 2nd dimension as 1), its 1st dimension must be the same as X, i.e. the number of data points and labels should be equal.eval_data (DataIter or numpy.ndarray/list/NDArray pair) – If eval_data is numpy.ndarray/list/NDArray pair, it should be (valid_data, valid_label).eval_metric (metric.EvalMetric or str or callable) – The evaluation metric, name of evaluation metric. Or a customize evaluation function that returns the statistics based on minibatch.epoch_end_callback (callable(epoch, symbol, arg_params, aux_states)) – A callback that is invoked at end of each epoch. This can be used to checkpoint model each epoch.batch_end_callback (callable(epoch)) – A callback that is invoked at end of each batch For print purposekvstore (KVStore or str, optional) – The KVStore or a string kvstore type: ‘local’, ‘dist_sync’, ‘dist_async’ In default uses ‘local’, often no need to change for single machiine.logger (logging logger, optional) – When not specified, default logger will be used.work_load_list (float or int, optional) – The list of work load for different devices, in the same order as ctx 类方法功能:模型拟合参数:X:训练集。Y:训练集标签。可以是二维的不过第二维是一标签的个数需要和输入点的个数一致。eval_data:解析数据(和javascript里面的eval函数差不多),输入应该是(vaild_data,vaild_label)eval_metric评估矩阵epoch_end_callback-在执行到每一epoch的结尾的时候调用。通常用来点检测。batch_end_callback-在每一批结尾都会调用只是为了打印出来看。kvstore:这个通常不用改基本上都是locallogger:当没有指定的时候会用默认的logger。work_load_list:不同设备的工作流列表和ctx的顺序一样。save(prefix, epochNone) Checkpoint the model checkpoint into file. You can also use pickle to do the job if you only work on python. The advantage of load/save is the file is language agnostic. This means the file saved using save can be loaded by other language binding of mxnet. You also get the benefit being able to directly load/save from cloud storage(S3, HDFS) Parameters:prefix (str) – Prefix of model name. Notes prefix-symbol.json will be saved for symbol.prefix-epoch.params will be saved for parameters. static load(prefix, epoch, ctxNone, **kwargs) Load model checkpoint from file. Parameters: prefix (str) – Prefix of model name.epoch (int) – epoch number of model we would like to load.ctx (Context or list of Context, optional) – The device context of training and prediction.kwargs (dict) – other parameters for model, including num_epoch, optimizer and numpy_batch_size Returns: model – The loaded model that can be used for prediction. Return type: FeedForward 保存和加载的比较简单我就不说了。static create(symbol, X, yNone, ctxNone, num_epochNone, epoch_sizeNone,optimizersgd, initializermxnet.initializer.Uniform object, eval_dataNone, eval_metricacc,epoch_end_callbackNone, batch_end_callbackNone, kvstorelocal, loggerNone,work_load_listNone, eval_batch_end_callbackNone, **kwargs)¶ Functional style to create a model. This function will be more consistent with functional languages such as R, where mutation is not allowed. Parameters: symbol (Symbol) – The symbol configuration of computation network.X (DataIter) – Training datay (numpy.ndarray, optional) – If X is numpy.ndarray y is required to setctx (Context or list of Context, optional) – The device context of training and prediction. To use multi GPU training, pass in a list of gpu contexts.num_epoch (int, optional) – Training parameter, number of training epochs(epochs).epoch_size (int, optional) – Number of batches in a epoch. In default, it is set to ceil(num_train_examples / batch_size)optimizer (str or Optimizer, optional) – Training parameter, name or optimizer object for training.initializier (initializer function, optional) – Training parameter, the initialization scheme used.eval_data (DataIter or numpy.ndarray pair) – If eval_set is numpy.ndarray pair, it should be (valid_data, valid_label)eval_metric (metric.EvalMetric or str or callable) – The evaluation metric, name of evaluation metric. Or a customize evaluation function that returns the statistics based on minibatch.epoch_end_callback (callable(epoch, symbol, arg_params, aux_states)) – A callback that is invoked at end of each epoch. This can be used to checkpoint model each epoch.batch_end_callback (callable(epoch)) – A callback that is invoked at end of each batch For print purposekvstore (KVStore or str, optional) – The KVStore or a string kvstore type: ‘local’, ‘dist_sync’, ‘dis_async’ In default uses ‘local’, often no need to change for single machiine.logger (logging logger, optional) – When not specified, default logger will be used.work_load_list (list of float or int, optional) – The list of work load for different devices, in the same order as ctx 创建模型这个API和前面也是大同小异。接下去的这些API不常用到 初使化的API参考 class mxnet.initializer.Initializer¶ Base class for Initializer. __call__(name, arr) Override () function to do Initialization Parameters: name (str) – name of corrosponding ndarrayarr (NDArray) – ndarray to be Initialized class mxnet.initializer.Load(param, default_initNone, verboseFalse) Initialize by loading pretrained param from file or dict Parameters: param (str or dict of str-NDArray) – param file or dict mapping name to NDArray.default_init (Initializer) – default initializer when name is not found in param.verbose (bool) – log source when initializing. class mxnet.initializer.Mixed(patterns, initializers) Initialize with mixed Initializer Parameters: patterns (list of str) – list of regular expression patterns to match parameter names.initializers (list of Initializer) – list of Initializer corrosponding to patterns class mxnet.initializer.Uniform(scale0.07) Initialize the weight with uniform [-scale, scale] Parameters:scale (float, optional) – The scale of uniform distribution class mxnet.initializer.Normal(sigma0.01) Initialize the weight with normal(0, sigma) Parameters:sigma (float, optional) – Standard deviation for gaussian distribution. class mxnet.initializer.Orthogonal(scale1.414, rand_typeuniform) Intialize weight as Orthogonal matrix Parameters: scale (float optional) – scaling factor of weightrand_type (string optional) – use “uniform” or “normal” random number to initialize weightReference –--------- –solutions to the nonlinear dynamics of learning in deep linear neural networks(Exact) –preprint arXiv (arXiv) – class mxnet.initializer.Xavier(rnd_typeuniform, factor_typeavg, magnitude3) Initialize the weight with Xavier or similar initialization scheme. Parameters: rnd_type (str, optional) – Use gaussian or uniform to initfactor_type (str, optional) – Use avg, in, or out to initmagnitude (float, optional) – scale of random number range 评估矩阵(Evalution Metric)APIOnline evaluation metric module. mxnet.metric.check_label_shapes(labels, preds, shape0) Check to see if the two arrays are the same size. class mxnet.metric.EvalMetric(name, numNone) Base class of all evaluation metrics. update(label, pred) Update the internal evaluation. Parameters: labels (list of NDArray) – The labels of the data.preds (list of NDArray) – Predicted values. reset() Clear the internal statistics to initial state. get() Get the current evaluation result. Returns: name (str) – Name of the metric.value (float) – Value of the evaluation. get_name_value() Get zipped name and value pairs class mxnet.metric.CompositeEvalMetric(**kwargs) Manage multiple evaluation metrics. add(metric) Add a child metric. get_metric(index) Get a child metric. class mxnet.metric.Accuracy Calculate accuracy class mxnet.metric.TopKAccuracy(**kwargs) Calculate top k predictions accuracy class mxnet.metric.F1 Calculate the F1 score of a binary classification problem. class mxnet.metric.MAE Calculate Mean Absolute Error loss class mxnet.metric.MSE Calculate Mean Squared Error loss class mxnet.metric.RMSE Calculate Root Mean Squred Error loss class mxnet.metric.CrossEntropy Calculate Cross Entropy loss class mxnet.metric.Torch Dummy metric for torch criterions class mxnet.metric.CustomMetric(feval, nameNone, allow_extra_outputsFalse) Custom evaluation metric that takes a NDArray function. Parameters: feval (callable(label, pred)) – Customized evaluation function.name (str, optional) – The name of the metricallow_extra_outputs (bool) – If true, the prediction outputs can have extra outputs. This is useful in RNN, where the states are also produced in outputs for forwarding. mxnet.metric.np(numpy_feval, nameNone, allow_extra_outputsFalse) Create a customized metric from numpy function. Parameters: numpy_feval (callable(label, pred)) – Customized evaluation function.name (str, optional) – The name of the metric.allow_extra_outputs (bool) – If true, the prediction outputs can have extra outputs. This is useful in RNN, where the states are also produced in outputs for forwarding. mxnet.metric.create(metric, **kwargs) Create an evaluation metric. Parameters:metric (str or callable) – The name of the metric, or a function providing statistics given pred, label NDArray 优化APICommon Optimization algorithms with regularizations. class mxnet.optimizer.Optimizer(rescale_grad1.0, param_idx2nameNone, wd0.0,clip_gradientNone, learning_rate0.01, lr_schedulerNone, symNone) Base class of all optimizers. static register(klass) Register optimizers to the optimizer factory static create_optimizer(name, rescale_grad1, **kwargs) Create an optimizer with specified name. Parameters: name (str) – Name of required optimizer. Should be the name of a subclass of Optimizer. Case insensitive.rescale_grad (float) – Rescaling factor on gradient.kwargs (dict) – Parameters for optimizer Returns: opt – The result optimizer. Return type: Optimizer create_state(index, weight) Create additional optimizer state such as momentum. override in implementations. update(index, weight, grad, state) Update the parameters. override in implementations set_lr_scale(args_lrscale) set lr scale is deprecated. Use set_lr_mult instead. set_lr_mult(args_lr_mult) Set individual learning rate multipler for parameters Parameters:args_lr_mult (dict of string/int to float) – set the lr multipler for name/index to float. setting multipler by index is supported for backward compatibility, but we recommend using name and symbol. set_wd_mult(args_wd_mult) Set individual weight decay multipler for parameters. By default wd multipler is 0 for all params whose name doesn’t end with _weight, if param_idx2name is provided. Parameters:args_wd_mult (dict of string/int to float) – set the wd multipler for name/index to float. setting multipler by index is supported for backward compatibility, but we recommend using name and symbol. mxnet.optimizer.register(klass) Register optimizers to the optimizer factory class mxnet.optimizer.SGD(momentum0.0, **kwargs) A very simple SGD optimizer with momentum and weight regularization. Parameters: learning_rate (float, optional) – learning_rate of SGDmomentum (float, optional) – momentum valuewd (float, optional) – L2 regularization coefficient add to all the weightsrescale_grad (float, optional) – rescaling factor of gradient.clip_gradient (float, optional) – clip gradient in range [-clip_gradient, clip_gradient]param_idx2name (dict of string/int to float, optional) – special treat weight decay in parameter ends with bias, gamma, and beta create_state(index, weight) Create additional optimizer state such as momentum. Parameters:weight (NDArray) – The weight data update(index, weight, grad, state) Update the parameters. Parameters: index (int) – An unique integer key used to index the parametersweight (NDArray) – weight ndarraygrad (NDArray) – grad ndarraystate (NDArray or other objects returned by init_state) – The auxiliary state used in optimization. class mxnet.optimizer.NAG(**kwargs) SGD with nesterov It is implemented according to https://github.com/torch/optim/blob/master/sgd.lua update(index, weight, grad, state) Update the parameters. Parameters: index (int) – An unique integer key used to index the parametersweight (NDArray) – weight ndarraygrad (NDArray) – grad ndarraystate (NDArray or other objects returned by init_state) – The auxiliary state used in optimization. class mxnet.optimizer.SGLD(**kwargs) Stochastic Langevin Dynamics Updater to sample from a distribution. Parameters: learning_rate (float, optional) – learning_rate of SGDwd (float, optional) – L2 regularization coefficient add to all the weightsrescale_grad (float, optional) – rescaling factor of gradient.clip_gradient (float, optional) – clip gradient in range [-clip_gradient, clip_gradient]param_idx2name (dict of string/int to float, optional) – special treat weight decay in parameter ends with bias, gamma, and beta create_state(index, weight) Create additional optimizer state such as momentum. Parameters:weight (NDArray) – The weight data update(index, weight, grad, state) Update the parameters. Parameters: index (int) – An unique integer key used to index the parametersweight (NDArray) – weight ndarraygrad (NDArray) – grad ndarraystate (NDArray or other objects returned by init_state) – The auxiliary state used in optimization. class mxnet.optimizer.ccSGD(momentum0.0, **kwargs) A very simple SGD optimizer with momentum and weight regularization. Implemented in C. Parameters: learning_rate (float, optional) – learning_rate of SGDmomentum (float, optional) – momentum valuewd (float, optional) – L2 regularization coefficient add to all the weightsrescale_grad (float, optional) – rescaling factor of gradient.clip_gradient (float, optional) – clip gradient in range [-clip_gradient, clip_gradient] update(index, weight, grad, state) Update the parameters. Parameters: index (int) – An unique integer key used to index the parametersweight (NDArray) – weight ndarraygrad (NDArray) – grad ndarraystate (NDArray or other objects returned by init_state) – The auxiliary state used in optimization. class mxnet.optimizer.Adam(learning_rate0.001, beta10.9, beta20.999, epsilon1e-08,decay_factor0.99999999, **kwargs) Adam optimizer as described in [King2014]. [King2014] Diederik Kingma, Jimmy Ba, Adam: A Method for Stochastic Optimization,http://arxiv.org/abs/1412.6980 the code in this class was adapted from https://github.com/mila-udem/blocks/blob/master/blocks/algorithms/__init__.py#L765 Parameters: learning_rate (float, optional) – Step size. Default value is set to 0.002.beta1 (float, optional) – Exponential decay rate for the first moment estimates. Default value is set to 0.9.beta2 (float, optional) – Exponential decay rate for the second moment estimates. Default value is set to 0.999.epsilon (float, optional) – Default value is set to 1e-8.decay_factor (float, optional) – Default value is set to 1 - 1e-8.wd (float, optional) – L2 regularization coefficient add to all the weightsrescale_grad (float, optional) – rescaling factor of gradient.clip_gradient (float, optional) – clip gradient in range [-clip_gradient, clip_gradient] create_state(index, weight) Create additional optimizer state: mean, variance Parameters:weight (NDArray) – The weight data update(index, weight, grad, state) Update the parameters. Parameters: index (int) – An unique integer key used to index the parametersweight (NDArray) – weight ndarraygrad (NDArray) – grad ndarraystate (NDArray or other objects returned by init_state) – The auxiliary state used in optimization. class mxnet.optimizer.AdaGrad(eps1e-07, **kwargs) AdaGrad optimizer of Duchi et al., 2011, This code follows the version in http://arxiv.org/pdf/1212.5701v1.pdf Eq(5) by Matthew D. Zeiler, 2012. AdaGrad will help the network to converge faster in some cases. Parameters: learning_rate (float, optional) – Step size. Default value is set to 0.05.wd (float, optional) – L2 regularization coefficient add to all the weightsrescale_grad (float, optional) – rescaling factor of gradient.eps (float, optional) – A small float number to make the updating processing stable Default value is set to 1e-7.clip_gradient (float, optional) – clip gradient in range [-clip_gradient, clip_gradient] class mxnet.optimizer.RMSProp(gamma10.95, gamma20.9, **kwargs) RMSProp optimizer of Tieleman Hinton, 2012, This code follows the version in http://arxiv.org/pdf/1308.0850v5.pdf Eq(38) - Eq(45) by Alex Graves, 2013. Parameters: learning_rate (float, optional) – Step size. Default value is set to 0.002.gamma1 (float, optional) – decay factor of moving average for gradient, gradient^2. Default value is set to 0.95.gamma2 (float, optional) – “momentum” factor. Default value if set to 0.9.wd (float, optional) – L2 regularization coefficient add to all the weightsrescale_grad (float, optional) – rescaling factor of gradient.clip_gradient (float, optional) – clip gradient in range [-clip_gradient, clip_gradient] create_state(index, weight) Create additional optimizer state: mean, variance :param weight: The weight data :type weight: NDArray update(index, weight, grad, state) Update the parameters. :param index: An unique integer key used to index the parameters Parameters: weight (NDArray) – weight ndarraygrad (NDArray) – grad ndarraystate (NDArray or other objects returned by init_state) – The auxiliary state used in optimization. class mxnet.optimizer.AdaDelta(rho0.9, epsilon1e-05, **kwargs) AdaDelta optimizer as described in Zeiler, M. D. (2012). ADADELTA: An adaptive learning rate method. http://arxiv.org/abs/1212.5701 Parameters: rho (float) – Decay rate for both squared gradients and delta xepsilon (float) – The constant as described in the thesiswd (float) – L2 regularization coefficient add to all the weightsrescale_grad (float, optional) – rescaling factor of gradient.clip_gradient (float, optional) – clip gradient in range [-clip_gradient, clip_gradient] class mxnet.optimizer.Test(**kwargs) For test use create_state(index, weight) Create a state to duplicate weight update(index, weight, grad, state) performs w rescale_grad * grad mxnet.optimizer.create(name, rescale_grad1, **kwargs) Create an optimizer with specified name. Parameters: name (str) – Name of required optimizer. Should be the name of a subclass of Optimizer. Case insensitive.rescale_grad (float) – Rescaling factor on gradient.kwargs (dict) – Parameters for optimizer Returns: opt – The result optimizer. Return type: Optimizer mxnet.optimizer.get_updater(optimizer) Return a clossure of the updater needed for kvstore Parameters:optimizer (Optimizer) – The optimizerReturns:updater – The clossure of the updaterReturn type:function
