Change search
Link to record
Permanent link

Direct link
BETA
Boström, Henrik
Publications (10 of 12) Show all publications
Johansson, U., Löfström, T., Sundell, H., Linnusson, H., Gidenstam, A. & Boström, H. (2018). Venn predictors for well-calibrated probability estimation trees. In: Alex J. Gammerman and Vladimir Vovk and Zhiyuan Luo and Evgueni N. Smirnov and Ralf L. M. Peeter (Ed.), 7th Symposium on Conformal and Probabilistic Prediction and Applications: COPA 2018, 11-13 June 2018, Maastricht, The Netherlands. Paper presented at 7th Symposium on Conformal and Probabilistic Prediction and Applications, London, June 11th - 13th, 2018 (pp. 3-14).
Open this publication in new window or tab >>Venn predictors for well-calibrated probability estimation trees
Show others...
2018 (English)In: 7th Symposium on Conformal and Probabilistic Prediction and Applications: COPA 2018, 11-13 June 2018, Maastricht, The Netherlands / [ed] Alex J. Gammerman and Vladimir Vovk and Zhiyuan Luo and Evgueni N. Smirnov and Ralf L. M. Peeter, 2018, p. 3-14Conference paper, Published paper (Refereed)
Abstract [en]

Successful use of probabilistic classification requires well-calibrated probability estimates, i.e., the predicted class probabilities must correspond to the true probabilities. The standard solution is to employ an additional step, transforming the outputs from a classifier into probability estimates. In this paper, Venn predictors are compared to Platt scaling and isotonic regression, for the purpose of producing well-calibrated probabilistic predictions from decision trees. The empirical investigation, using 22 publicly available datasets, showed that the probability estimates from the Venn predictor were extremely well-calibrated. In fact, in a direct comparison using the accepted reliability metric, the Venn predictor estimates were the most exact on every data set.

Series
Proceedings of Machine Learning Research
Keywords
Venn predictors, Calibration, Decision trees, Reliability
National Category
Computer Sciences
Research subject
Business and IT
Identifiers
urn:nbn:se:hb:diva-15061 (URN)
Conference
7th Symposium on Conformal and Probabilistic Prediction and Applications, London, June 11th - 13th, 2018
Funder
Knowledge Foundation
Available from: 2018-09-04 Created: 2018-09-04 Last updated: 2020-01-29Bibliographically approved
Linusson, H., Norinder, U., Boström, H., Johansson, U. & Löfström, T. (2017). On the Calibration of Aggregated Conformal Predictors. In: Proceedings of Machine Learning Research: . Paper presented at Conformal and Probabilistic Prediction and Applications, Stockholm Sweden 13-16 June, 2017.
Open this publication in new window or tab >>On the Calibration of Aggregated Conformal Predictors
Show others...
2017 (English)In: Proceedings of Machine Learning Research, 2017Conference paper, Published paper (Refereed)
Abstract [en]

Conformal prediction is a learning framework that produces models that associate witheach of their predictions a measure of statistically valid confidence. These models are typi-cally constructed on top of traditional machine learning algorithms. An important result ofconformal prediction theory is that the models produced are provably valid under relativelyweak assumptions—in particular, their validity is independent of the specific underlyinglearning algorithm on which they are based. Since validity is automatic, much research onconformal predictors has been focused on improving their informational and computationalefficiency. As part of the efforts in constructing efficient conformal predictors, aggregatedconformal predictors were developed, drawing inspiration from the field of classification andregression ensembles. Unlike early definitions of conformal prediction procedures, the va-lidity of aggregated conformal predictors is not fully understood—while it has been shownthat they might attain empirical exact validity under certain circumstances, their theo-retical validity is conditional on additional assumptions that require further clarification.In this paper, we show why validity is not automatic for aggregated conformal predictors,and provide a revised definition of aggregated conformal predictors that gains approximatevalidity conditional on properties of the underlying learning algorithm.

National Category
Computer Sciences
Identifiers
urn:nbn:se:hb:diva-13636 (URN)
Conference
Conformal and Probabilistic Prediction and Applications, Stockholm Sweden 13-16 June, 2017
Available from: 2018-02-09 Created: 2018-02-09 Last updated: 2020-01-29Bibliographically approved
Linusson, H., Johansson, U., Boström, H. & Löfström, T. (2014). Efficiency Comparison of Unstable Transductive and Inductive Conformal Classifiers. Paper presented at Artificial Intelligence Applications and Innovations. Paper presented at Artificial Intelligence Applications and Innovations. Springer
Open this publication in new window or tab >>Efficiency Comparison of Unstable Transductive and Inductive Conformal Classifiers
2014 (English)Conference paper, Published paper (Refereed)
Abstract [en]

In the conformal prediction literature, it appears axiomatic that transductive conformal classifiers possess a higher predictive efficiency than inductive conformal classifiers, however, this depends on whether or not the nonconformity function tends to overfit misclassified test examples. With the conformal prediction framework’s increasing popularity, it thus becomes necessary to clarify the settings in which this claim holds true. In this paper, the efficiency of transductive conformal classifiers based on decision tree, random forest and support vector machine classification models is compared to the efficiency of corresponding inductive conformal classifiers. The results show that the efficiency of conformal classifiers based on standard decision trees or random forests is substantially improved when used in the inductive mode, while conformal classifiers based on support vector machines are more efficient in the transductive mode. In addition, an analysis is presented that discusses the effects of calibration set size on inductive conformal classifier efficiency.

Place, publisher, year, edition, pages
Springer, 2014
Series
IFIP Advances in Information and Communication Technology, ISSN 1868-4238 ; 437
Keywords
Conformal Prediction, Machine learning, Data mining
National Category
Computer Sciences Computer and Information Sciences
Identifiers
urn:nbn:se:hb:diva-7323 (URN)10.1007/978-3-662-44722-2_28 (DOI)2320/14626 (Local ID)978-3-662-44721-5 (ISBN)978-3-662-44722-2 (ISBN)2320/14626 (Archive number)2320/14626 (OAI)
Conference
Artificial Intelligence Applications and Innovations
Note

Sponsorship:

This work was supported by the Swedish Foundation

for Strategic Research through the project High-Performance Data Mining for

Drug Effect Detection (IIS11-0053) and the Knowledge Foundation through the

project Big Data Analytics by Online Ensemble Learning (20120192).

Available from: 2015-12-22 Created: 2015-12-22 Last updated: 2020-01-29
Jansson, K., Sundell, H. & Boström, H. (2014). gpuRF and gpuERT: Efficient and Scalable GPU Algorithms for Decision Tree Ensembles. In: Parallel & Distributed Processing Symposium Workshops (IPDPSW), 2014 IEEE International: . Paper presented at Parallel & Distributed Processing Symposium Workshops (IPDPSW), 2014 IEEE International, May 19-23, 2014. (pp. 1612-1621).
Open this publication in new window or tab >>gpuRF and gpuERT: Efficient and Scalable GPU Algorithms for Decision Tree Ensembles
2014 (English)In: Parallel & Distributed Processing Symposium Workshops (IPDPSW), 2014 IEEE International, 2014, p. 1612-1621Conference paper, Published paper (Refereed)
National Category
Computer Sciences
Identifiers
urn:nbn:se:hb:diva-4063 (URN)
Conference
Parallel & Distributed Processing Symposium Workshops (IPDPSW), 2014 IEEE International, May 19-23, 2014.
Available from: 2015-12-15 Created: 2015-12-15 Last updated: 2018-06-14Bibliographically approved
Johansson, U., Boström, H., Löfström, T. & Linusson, H. (2014). Regression conformal prediction with random forests. Machine Learning, 97(1-2), 155-176
Open this publication in new window or tab >>Regression conformal prediction with random forests
2014 (English)In: Machine Learning, ISSN 0885-6125, E-ISSN 1573-0565, Vol. 97, no 1-2, p. 155-176Article in journal (Refereed)
Abstract [en]

Regression conformal prediction produces prediction intervals that are valid, i.e., the probability of excluding the correct target value is bounded by a predefined confidence level. The most important criterion when comparing conformal regressors is efficiency; the prediction intervals should be as tight (informative) as possible. In this study, the use of random forests as the underlying model for regression conformal prediction is investigated and compared to existing state-of-the-art techniques, which are based on neural networks and k-nearest neighbors. In addition to their robust predictive performance, random forests allow for determining the size of the prediction intervals by using out-of-bag estimates instead of requiring a separate calibration set. An extensive empirical investigation, using 33 publicly available data sets, was undertaken to compare the use of random forests to existing stateof- the-art conformal predictors. The results show that the suggested approach, on almost all confidence levels and using both standard and normalized nonconformity functions, produced significantly more efficient conformal predictors than the existing alternatives.

Place, publisher, year, edition, pages
Springer New York LLC, 2014
Keywords
Conformal prediction, Random forests, Regression, Machine learning, Data mining
National Category
Computer Sciences Computer and Information Sciences
Identifiers
urn:nbn:se:hb:diva-2030 (URN)10.1007/s10994-014-5453-0 (DOI)2320/14623 (Local ID)2320/14623 (Archive number)2320/14623 (OAI)
Note

Sponsorship:

This

work was supported by the Swedish Foundation for Strategic Research through the project High-Performance

Data Mining for Drug Effect Detection (IIS11-0053) and the Knowledge Foundation through the project Big

Data Analytics by Online Ensemble Learning (20120192).

Available from: 2015-11-13 Created: 2015-11-13 Last updated: 2020-01-29
Johansson, U., Sönströd, C., Linusson, H. & Boström, H. (2014). Regression Trees for Streaming Data with Local Performance Guarantees. Paper presented at IEEE International Conference on Big Data, 27-30 October, 2014, Washington, DC, USA. Paper presented at IEEE International Conference on Big Data, 27-30 October, 2014, Washington, DC, USA. IEEE
Open this publication in new window or tab >>Regression Trees for Streaming Data with Local Performance Guarantees
2014 (English)Conference paper, Published paper (Refereed)
Abstract [en]

Online predictive modeling of streaming data is a key task for big data analytics. In this paper, a novel approach for efficient online learning of regression trees is proposed, which continuously updates, rather than retrains, the tree as more labeled data become available. A conformal predictor outputs prediction sets instead of point predictions; which for regression translates into prediction intervals. The key property of a conformal predictor is that it is always valid, i.e., the error rate, on novel data, is bounded by a preset significance level. Here, we suggest applying Mondrian conformal prediction on top of the resulting models, in order to obtain regression trees where not only the tree, but also each and every rule, corresponding to a path from the root node to a leaf, is valid. Using Mondrian conformal prediction, it becomes possible to analyze and explore the different rules separately, knowing that their accuracy, in the long run, will not be below the preset significance level. An empirical investigation, using 17 publicly available data sets, confirms that the resulting rules are independently valid, but also shows that the prediction intervals are smaller, on average, than when only the global model is required to be valid. All-in-all, the suggested method provides a data miner or a decision maker with highly informative predictive models of streaming data.

Place, publisher, year, edition, pages
IEEE, 2014
Keywords
Conformal Prediction, Streaming data, Regression trees, Interpretable models, Machine learning, Data mining
National Category
Computer Sciences Computer and Information Sciences
Identifiers
urn:nbn:se:hb:diva-7324 (URN)10.1109/BigData.2014.7004263 (DOI)2320/14627 (Local ID)978-1-4799-5666-1/14 (ISBN)2320/14627 (Archive number)2320/14627 (OAI)
Conference
IEEE International Conference on Big Data, 27-30 October, 2014, Washington, DC, USA
Note

Sponsorship:

This work was supported by the Swedish Foundation for Strategic

Research through the project High-Performance Data Mining for Drug Effect

Detection (IIS11-0053), the Swedish Retail and Wholesale Development

Council through the project Innovative Business Intelligence Tools (2013:5)

and the Knowledge Foundation through the project Big Data Analytics by

Online Ensemble Learning (20120192).

Available from: 2015-12-22 Created: 2015-12-22 Last updated: 2018-01-10
Johansson, U., König, R., Linusson, H., Löfström, T. & Boström, H. (2014). Rule Extraction with Guaranteed Fidelity. Paper presented at Artificial Intelligence Applications and Innovations. Paper presented at Artificial Intelligence Applications and Innovations. Springer
Open this publication in new window or tab >>Rule Extraction with Guaranteed Fidelity
Show others...
2014 (English)Conference paper, Published paper (Refereed)
Abstract [en]

This paper extends the conformal prediction framework to rule extraction, making it possible to extract interpretable models from opaque models in a setting where either the infidelity or the error rate is bounded by a predefined significance level. Experimental results on 27 publicly available data sets show that all three setups evaluated produced valid and rather efficient conformal predictors. The implication is that augmenting rule extraction with conformal prediction allows extraction of models where test set errors or test sets infidelities are guaranteed to be lower than a chosen acceptable level. Clearly this is beneficial for both typical rule extraction scenarios, i.e., either when the purpose is to explain an existing opaque model, or when it is to build a predictive model that must be interpretable.

Place, publisher, year, edition, pages
Springer, 2014
Series
IFIP Advances in Information and Communication Technology, ISSN 1868-4238 ; 437
Keywords
Rule extraction, Conformal Prediction, Decision trees, Machine learning, Data mining
National Category
Computer Sciences Computer and Information Sciences
Identifiers
urn:nbn:se:hb:diva-7322 (URN)10.1007/978-3-662-44722-2_30 (DOI)2320/14625 (Local ID)978-3-662-44721-5 (ISBN)978-3-662-44722-2 (ISBN)2320/14625 (Archive number)2320/14625 (OAI)
Conference
Artificial Intelligence Applications and Innovations
Note

Sponsorship:

This work was supported by the Swedish Foundation for Strategic Research through

the project High-Performance Data Mining for Drug Effect Detection (IIS11-0053)

and the Knowledge Foundation through the project Big Data Analytics by Online

Ensemble Learning (20120192).

Available from: 2015-12-22 Created: 2015-12-22 Last updated: 2020-01-29
Johansson, U., Boström, H. & Löfström, T. (2013). Conformal Prediction Using Decision Trees. Paper presented at IEEE International Conference on Data Mining. Paper presented at IEEE International Conference on Data Mining. IEEE
Open this publication in new window or tab >>Conformal Prediction Using Decision Trees
2013 (English)Conference paper, Published paper (Refereed)
Abstract [en]

Conformal prediction is a relatively new framework in which the predictive models output sets of predictions with a bound on the error rate, i.e., in a classification context, the probability of excluding the correct class label is lower than a predefined significance level. An investigation of the use of decision trees within the conformal prediction framework is presented, with the overall purpose to determine the effect of different algorithmic choices, including split criterion, pruning scheme and way to calculate the probability estimates. Since the error rate is bounded by the framework, the most important property of conformal predictors is efficiency, which concerns minimizing the number of elements in the output prediction sets. Results from one of the largest empirical investigations to date within the conformal prediction framework are presented, showing that in order to optimize efficiency, the decision trees should be induced using no pruning and with smoothed probability estimates. The choice of split criterion to use for the actual induction of the trees did not turn out to have any major impact on the efficiency. Finally, the experimentation also showed that when using decision trees, standard inductive conformal prediction was as efficient as the recently suggested method cross-conformal prediction. This is an encouraging results since cross-conformal prediction uses several decision trees, thus sacrificing the interpretability of a single decision tree.

Place, publisher, year, edition, pages
IEEE, 2013
Keywords
Conformal prediction, Decision trees, Data mining, Machine Learning
National Category
Computer Sciences Computer and Information Sciences
Identifiers
urn:nbn:se:hb:diva-7055 (URN)10.1109/ICDM.2013.85 (DOI)000332874200034 ()2320/13010 (Local ID)2320/13010 (Archive number)2320/13010 (OAI)
Conference
IEEE International Conference on Data Mining
Note

Sponsorship:

Swedish Foundation

for Strategic Research through the project High-Performance

Data Mining for Drug Effect Detection (IIS11-0053) and the

Knowledge Foundation through the project Big Data Analytics

by Online Ensemble Learning (20120192)

Available from: 2015-12-22 Created: 2015-12-22 Last updated: 2020-01-29
Löfström, T., Johansson, U. & Boström, H. (2013). Effective Utilization of Data in Inductive Conformal Prediction. Paper presented at International Joint Conference on Neural Networks, Dallas, TX, USA, August 4-9, 2013.. Paper presented at International Joint Conference on Neural Networks, Dallas, TX, USA, August 4-9, 2013.. IEEE
Open this publication in new window or tab >>Effective Utilization of Data in Inductive Conformal Prediction
2013 (English)Conference paper, Published paper (Refereed)
Abstract [en]

Conformal prediction is a new framework producing region predictions with a guaranteed error rate. Inductive conformal prediction (ICP) was designed to significantly reduce the computational cost associated with the original transductive online approach. The drawback of inductive conformal prediction is that it is not possible to use all data for training, since it sets aside some data as a separate calibration set. Recently, cross-conformal prediction (CCP) and bootstrap conformal prediction (BCP) were proposed to overcome that drawback of inductive conformal prediction. Unfortunately, CCP and BCP both need to build several models for the calibration, making them less attractive. In this study, focusing on bagged neural network ensembles as conformal predictors, ICP, CCP and BCP are compared to the very straightforward and cost-effective method of using the out-of-bag estimates for the necessary calibration. Experiments on 34 publicly available data sets conclusively show that the use of out-of-bag estimates produced the most efficient conformal predictors, making it the obvious preferred choice for ensembles in the conformal prediction framework.

Place, publisher, year, edition, pages
IEEE, 2013
Keywords
Data mining, Machine Learning
National Category
Computer Sciences Computer and Information Sciences
Identifiers
urn:nbn:se:hb:diva-7058 (URN)2320/12923 (Local ID)2320/12923 (Archive number)2320/12923 (OAI)
Conference
International Joint Conference on Neural Networks, Dallas, TX, USA, August 4-9, 2013.
Note

Sponsorship:

Swedish Foundation for Strategic

Research through the project High-Performance Data Mining for Drug Effect

Detection (IIS11-0053) and the Knowledge Foundation through the project

Big Data Analytics by Online Ensemble Learning (20120192)

Available from: 2015-12-22 Created: 2015-12-22 Last updated: 2020-01-29
Johansson, U., König, R., Löfström, T. & Boström, H. (2013). Evolved Decision Trees as Conformal Predictors. Paper presented at IEEE Congress on Evolutionary Computation, 20-23 June 2013. Paper presented at IEEE Congress on Evolutionary Computation, 20-23 June 2013. IEEE
Open this publication in new window or tab >>Evolved Decision Trees as Conformal Predictors
2013 (English)Conference paper, Published paper (Refereed)
Abstract [en]

In conformal prediction, predictive models output sets of predictions with a bound on the error rate. In classification, this translates to that the probability of excluding the correct class is lower than a predefined significance level, in the long run. Since the error rate is guaranteed, the most important criterion for conformal predictors is efficiency. Efficient conformal predictors minimize the number of elements in the output prediction sets, thus producing more informative predictions. This paper presents one of the first comprehensive studies where evolutionary algorithms are used to build conformal predictors. More specifically, decision trees evolved using genetic programming are evaluated as conformal predictors. In the experiments, the evolved trees are compared to decision trees induced using standard machine learning techniques on 33 publicly available benchmark data sets, with regard to predictive performance and efficiency. The results show that the evolved trees are generally more accurate, and the corresponding conformal predictors more efficient, than their induced counterparts. One important result is that the probability estimates of decision trees when used as conformal predictors should be smoothed, here using the Laplace correction. Finally, using the more discriminating Brier score instead of accuracy as the optimization criterion produced the most efficient conformal predictions.

Place, publisher, year, edition, pages
IEEE, 2013
Keywords
Conformal prediction, Genetic programming, Data mining, Machine Learning
National Category
Computer Sciences Computer and Information Sciences
Identifiers
urn:nbn:se:hb:diva-7053 (URN)10.1109/CEC.2013.6557778 (DOI)000326235301102 ()2320/12919 (Local ID)978-1-4799-0453-2 (ISBN)2320/12919 (Archive number)2320/12919 (OAI)
Conference
IEEE Congress on Evolutionary Computation, 20-23 June 2013
Note

Sponsorship:

Swedish Foundation

for Strategic Research through the project High-Performance

Data Mining for Drug Effect Detection (IIS11-0053) and the

Knowledge Foundation through the project Big Data Analytics

by Online Ensemble Learning (20120192).

Available from: 2015-12-22 Created: 2015-12-22 Last updated: 2020-01-29
Organisations

Search in DiVA

Show all publications