How to Use TALENT

This guide will walk you through how to install, set up and use the TALENT toolbox for benchmarking models on tabular data, running experiments, and adding new methods.

Installation

You can install TALENT directly from GitHub:

pip install git+https://github.com/LAMDA-Tabular/TALENT.git@main --upgrade

Alternatively, for development purposes you can clone the repository:

git clone https://github.com/LAMDA-Tabular/TALENT
cd TALENT/test

2. Quick Start

Here’s a basic example of how to run a deep learning model experiment:

from tqdm import tqdm
from TALENT.model.utils import get_deep_args,show_results,tune_hyper_parameters,get_method,set_seeds
from TALENT.model.lib.data import get_dataset

if __name__ == '__main__':
loss_list, results_list, time_list = [], [], []
args,default_para,opt_space = get_deep_args()
train_val_data,test_data,info = get_dataset(args.dataset,args.dataset_path)
if args.tune:
     args = tune_hyper_parameters(args,opt_space,train_val_data,info)
for seed in tqdm(range(args.seed_num)):
     args.seed = seed # update seed
     set_seeds(args.seed)
     method = get_method(args.model_type)(args, info['task_type'] == 'regression')
     time_cost = method.fit(train_val_data, info)
     vl, vres, metric_name, predict_logits = method.predict(test_data, info, model_name=args.evaluate_option)
     loss_list.append(vl)
     results_list.append(vres)
     time_list.append(time_cost)

show_results(args,info, metric_name,loss_list,results_list,time_list)

Run the script from command line:

python train_model_deep.py --model_type MODEL_NAME

3. Running Experiments

TALENT supports running experiments for both deep learning methods and classical machine learning models:

Configure the experiment settings:

Edit the configuration files located in configs/default/[MODEL_NAME].json and configs/opt_space/[MODEL_NAME].json

Run the experiment:

For deep learning methods:

python train_model_deep.py --model_type [MODEL_NAME]

For classical machine learning methods:

python train_model_classical.py --model_type [MODEL_NAME]

4. Adding New Methods

For methods that only need model design:

Add the model class to model/models/

Inherit from model/methods/base.py and override construct_model()

Add the method name in get_method function in model/utils.py

Add parameter settings in configs/default/[MODEL_NAME].json and configs/opt_space/[MODEL_NAME].json

For methods requiring training process changes, partially override functions based on model/methods/base.py. Refer to existing implementations in model/methods/.

5. Configuring Hyperparameters

Hyperparameters can be configured through:

configs/default/: Default parameters for each method

configs/opt_space/: Hyperparameter optimization space

Modify the appropriate .json files to adjust parameters like learning rate, batch size, etc.

6. Troubleshooting

If you encounter any issues while using TALENT, try the following steps:

  1. Check the logs: Review the logs in the logs/ directory for any error messages.

  2. Verify dependencies: Ensure that all required dependencies are installed. Refer to the dependencies.rst for more information.

  3. Configuration issues: Double-check your configuration files to ensure the paths, dataset names, and hyperparameters are correct.

  4. Contact: If you’re unable to resolve the issue, feel free to open an issue on GitHub or contact the developers.

Conclusion

TALENT provides a flexible and powerful platform for experimenting with both classical and deep learning models on tabular data. By following the steps in this guide, you can quickly set up and run experiments, fine-tune models, and even add your own methods to the toolbox. For any further assistance, refer to the documentation or reach out to the development team.