|
Abstract:
|
Machine learning (ML) and Artificial Intelligence (AI) technologies have fueled unprecedented levels of interest across the financial industry. One area of significance is the application of Natural Language Processing techniques in FinTech, which has many different use cases, such as credit scoring for lending. Deep learning models with neural networks are commonly used in such applications, and different metrics can be used to track performance.
However, in real world applications, such performance tracking comes with many challenges, from the ambiguity of language and quality of the labeled data, to variations across multiple data sources, as well as model architecture and model hyper-parameters. In this discussion, we will demonstrate an example with simulated SMS messages, and illustrate the challenges in evaluating such models in real life applications from both technical and industry perspectives. We will also discuss some best practices in tracking performance with statistical tools and procedures.
|
