Deep Reinforcement and Transfer Learning for Abstractive Text Summarization: A Review
Authors : Ayham Alomari, Norisma Idris, Aznul Sabri, Izzat Alsmadi
Abstract : Automatic Text Summarization (ATS) is an important area in Natural Language Processing (NLP) with the goal of shortening a long text into a more compact version by conveying the most important points in a readable form. ATS applications continue to evolve and utilize effective approaches that are being evaluated and implemented by researchers. State-of-the-Art (SotA) technologies that demonstrate cutting-edge performance and accuracy in abstractive ATS are deep neural sequence-to-sequence models, Reinforcement Learning (RL) approaches, and Transfer Learning (TL) approaches, including Pre-Trained Language Models (PTLMs). The graph-based Transformer architecture and PTLMs have influenced tremendous advances in NLP applications. Additionally, the incorporation of recent mechanisms, such as the knowledge-enhanced mechanism, significantly enhanced the results. This study provides a comprehensive review of recent research advances in the area of abstractive text summarization for works spanning the past six years. Past and present problems are described, as well as their proposed solutions. In addition, abstractive ATS datasets and evaluation measurements are also highlighted. The paper concludes by comparing the best models and discussing future research directions.
Keywords : Abstractive summarization; Sequence-to-sequence; Reinforcement learning; Pre-trained models
رابط البحث
Improving Novelty of Abstractive Summarization via Warm-Starting
Authors : Ayham Alomari, Ahmad Sami Al-Shamayleh, Norisma Idris, Aznul Qalid Md Sabri, Izzat Alsmadi
Abstract : Abstractive summarization is distinguished by using novel phrases that are not found in the source text. However, most previous research ignores this feature in favour of enhancing syntactical similarity with the reference. To improve novelty, we warm-started various models with varying encoder and decoder checkpoints and vocabulary. These models are then adapted to the paraphrasing task and the sampling decoding strategy to further boost the levels of novelty and quality. In addition, to avoid relying only on syntactical similarity assessment, two additional abstractive summarization metrics are introduced which are: 1) NovScore: a new novelty metric that delivers a summary novelty score; and 2) NSSF: a new comprehensive metric that ensembles Novelty, Syntactic, Semantic, and Faithfulness features into a single score to simulate human assessment in providing a reliable evaluation. Finally, we compare our models to the state-of-the-art sequence-to-sequence models using the current and the proposed metrics. As a result, warm-starting, sampling, and paraphrasing improve novelty degrees by 2%, 5%, and 14%, respectively, while maintaining comparable scores on other metrics.
Keywords : Abstractive Summarization; Novelty; Warm-Started Models; Deep Learning; Metrics
رابط البحث
Improving Coverage and Novelty of Abstractive Text Summarization Using Transfer Learning and Divide and Conquer Approaches
Authors : Ayham Alomari, Norisma Idris, Aznul Sabri, Izzat Alsmadi
Abstract : Automatic Text Summarization (ATS) models yield outcomes with insufficient coverage of crucial details and poor degrees of novelty. The first issue resulted from the lengthy input, while the second problem resulted from the characteristics of the training dataset itself. This research employs the divide-and-conquer approach to address the first issue by breaking the lengthy input into smaller pieces to be summarized, followed by the conquest of the results in order to cover more significant details. For the second challenge, these chunks are summarized by models trained on datasets with higher novelty levels in order to produce more human-like and concise summaries with more novel words that do not appear in the input article. The results demonstrate an improvement in both coverage and novelty levels. Moreover, we defined a new metric to measure the novelty of the summary. Finally, the findings led us to conclude that the novelty levels are more significantly influenced by the training dataset itself, as in CNN/DM, than by other factors like the training model or its training objective, as in Pegasus.
Keywords : Abstractive Summarization, Novelty, Coverage, Warm-Started Models, Transfer Learning, Deep Learning
رابط البحث
An interactive predictive system for weather forecasting
Authors : Ayham Alomari, Ahmad Wedyan, Ahmed Zghoul, Ahmad Banihani, Izzat Alsmad
Abstract : Studying precipitation and weather data using Artificial Intelligent (AI) and data mining techniques has been the subject of several research papers. In this paper, a dataset is built about Jordanian weather and precipitation related information. This information is gathered from local and web resources. A tool is built to parse all weather related information from different websites that store such information. Data mining techniques and AI algorithms are used for future precipitation forecasting based on historical data. Data mining and statistical methods are used to predict future forecasting and possible climate change.
Keywords : Weather forecasting, data mining, and prediction algorithms, Numerical Weather Prediction models.
رابط البحث