OpenML is integrated in the Weka (Waikato Environment for Knowledge Analysis) Experimenter and the Command Line Interface.
The current beta integration is available as a stand alone WEKA version which can be downloaded here:
Quick Start Experimenter
Open WEKA allows you to run OpenML Tasks in the Weka Experimenter. You can solve OpenML tasks locally and/or automatically upload your experiments to OpenML.
Create an account on OpenML.org (click the user icon in the top bar). You need this only if you want to upload your results.
Download the extended WEKA environment by clicking on the Download button above. Open the jar file.
After starting Weka, choose the 'Experimenter' from the GUI Chooser.
In the Weka Experimenter, click the "New" button. (For the moment this is the only option, a more dedicated GUI will be released in the near future.)
If you want to upload experiments to OpenML.org, choose 'OpenML.org' under 'Results Destination'. Connect to your OpenML account using the 'Login' button.
The Experiment Type should now be "OpenML Task". All other experimenter inputs should be disabled (they are defined in OpenML tasks).
In the "Tasks" panel, click the "Add New" button to add new Tasks. Insert the task id's as comma-separated values (e.g., '1,2,3,4,5'). Use this form to search for interesting tasks. In the future this will also be integrated in WEKA.
Add algorithms in the "Algorithm" panel.
Go to the "Run" tab, and click on the "Start" button.
The experiment will be executed, and if indicated, also sent to OpenML.org. When the experiment is finished, the results can be inspected in the "Analyse" tab.
In your browser, log in to OpenML.org. Click on your name and choose 'My runs' to see a list of all submitted runs. They can now be queried together with all OpenML results. More overviews of your personal experiments will be added soon.
Quick Start CommandLine interface
The Command Line interface is useful for running experiments automatically on a server, without the possibility of invoking a GUI.
Make sure a recent version of JRE is installed (version 1.6 or higher).
Open a console and browse to the same directory as the Weka JAR.
Create a config file called openml.conf. This config file should be in the same directory as the Weka jar.
This config file should contain two lines: Line 1 contains a string in the format username = <Your username>. Line 2 contains a string in the format password = <Your password>