Exploring the Influence of Focal Loss on Transformer Models for Imbalanced Maintenance Data in Industry 4.0 - Université Polytechnique des Hauts-de-France Access content directly
Conference Papers Year :

Exploring the Influence of Focal Loss on Transformer Models for Imbalanced Maintenance Data in Industry 4.0

Juan Pablo Usuga Cadavid
  • Function : Author
  • PersonId : 1102943
Bernard Grabot
  • Function : Author
Arnaud Fortin
  • Function : Author
  • PersonId : 1102944

Abstract

Harnessing data from historical maintenance databases may be challenging, as they tend to rely on text data provided by operators. Thus, they often include acronyms, jargon, typos, and other irregularities that complicate the automated analysis of such reports. Furthermore, maintenance datasets may present highly imbalanced distributions: some situations happen more often than others, which hinders the effective application of classic Machine Learning (ML) models. Hence, this paper explores the use of a recent Deep Learning (DL) architecture called Transformer, which has provided cutting-edge results in Natural Language Processing (NLP). To tackle the class imbalance, a loss function called Focal Loss (FL) is explored. Results suggests that when all the classes are equally important, the FL does not improve the classification performance. However, if the objective is to detect the minority class, the FL achieves the best performance, although by degrading the detection capacity for the majority class.
No file

Dates and versions

hal-03407955 , version 1 (28-10-2021)

Identifiers

  • HAL Id : hal-03407955 , version 1

Cite

Juan Pablo Usuga Cadavid, Bernard Grabot, Samir Lamouri, Arnaud Fortin. Exploring the Influence of Focal Loss on Transformer Models for Imbalanced Maintenance Data in Industry 4.0. 17th IFAC Symposium on Information Control Problems in Manufacturing (INCOM 2021), Jun 2021, Budapest, Hungary. ⟨hal-03407955⟩
28 View
0 Download

Share

Gmail Facebook Twitter LinkedIn More