climin: optimization, straight-forward

introduction

Optimization is a key ingredient in modern machine learning. While many models can be optimized in specific ways, several need very general gradient based techniques–e.g. neural networks. What’s even worse is that you never know whether your model does not work or you just have not found the right optimizer.

This is where climin comes in. We offer you many different off-the-shelf optimizers, such as LBFGS, stochastic gradient descent with Nesterov momentum, nonlinear conjugate gradients, resilient propagation, rmsprop and more.

But what is best, is that we know that optimization is a process that needs to be analyzed. This is why climin does not offer you a black box function call that takes ages to run and might give you a good minimum of your training loss. Since this is not what you care about, climin takes you with you on its travel through the error landscape... in a classic for loop:

import climin

network = make_network()                 # your neural network
training_data = load_train_data()        # your training data
validation_data = load_validation_data() # your validation data
test_data = load_test_data()             # your testing data

opt = climin.Lbfgs(network.parameters,
                   network.loss,
                   network.d_loss_d_parameters,
                   args=itertools.repeat((training_data, {})))

for info in opt:
    validation_loss = network.loss(network.parameters, validation_data)
    print info['loss'], validation_loss

print network.loss(test_data)

Climin works on the CPU (via numpy and scipy) and in parts on the GPU (via gnumpy).

Starting points

If you want to see how climin works and use climin asap, check out the Tutorial. Details on Installation are available. A list of the optimizers implemented can be found in the overview below. If you want to understand the design decisions of climin, read the Manifest.

Contact & Community

Climin was started by people from the Biomimetic robotics and machine learning group at the Technische Universitaet Muenchen. If you have any questions, there is a mailinglist at climin@librelist.com. Just send an email to subscribe or checkout the archive. The software is hosted over at our github repository, where you can also find our issue tracker.

Indices and tables