iSOUP-SymRF experiments

This document details how to reproduce the experiments in the paper "iSOUP-SymRF: Symbolic feature ranking with random forests in online multi-target regression and multi-label classification".

The required files are available here.

Step 0: Preparing MOA

Acquire the MOA source code from github. Replace the ISOUPTreeRF.java in moa/moa/src/main/java/moa/classifiers/multilabel/trees with the one in the zip file provided above. This adds the feature ranking functionality to random forests of iSOUP-Trees. Build MOA.

This step will become obsolete once the source is merged.

Step 1: Acquire datasets

Datasets are available at:

• [MTR] Bicycles
• [MTR] SCM1d and SCM20d
• [MTR] Mars Express, download the 60 minute dataset variant
• [MLC] Hypercubes
• [MLC] Hyperspehres
• [MLC] Mediamill and TMC

Step 2: Configure experiments

Setup the experiments in the src folder in the zip file provided above. The FeatureRanker class calculates the feature scores for all the datasets, while the FeatureRankerFFAMTR and FeatureRankerFFAMLC provide utility experiments for MTR and MLC, respectively. Configure the proper paths in all experiments.

Step 3: Run experiments

Run the FeatureRanker experiments for seeds 1000, 2000, ..., 10000. This will yield scores for all datasets. After this is complete run FeatureRankerFFAMTR and FeatureRankerFFAMLC for seed 1000. This will provide predictions for the utility experiments.

Step 4: Post-process the results

Run the two python notebooks process_mtr.ipynb and process_mlc.ipynb from the zip file above, which generate the tables and plots shown in the paper.