دمج تقنية CNN-LSTM لتحسين التعرف على الكلام النظيف والصاخب
##plugins.themes.bootstrap3.article.sidebar##
منشور:
Dec 30, 2024
الكلمات المفتاحية:
التعرف الآلي على الكلام CNN - - LSTM -الكلام النظيف - الكلام الصاخب - - CNN -LSTM - DNN .SNR
##plugins.themes.bootstrap3.article.main##
الملخص
تخص هذه الورقة البحثية نهجًا هجينًا للشبكة العصبية التلافيفية والذاكرة طويلة المدى القصيرة (CNN-LSTM) للتعرف الآلي على الكلام (ASR) باستخدام تقنيات التعلم العميق على قاعدة بيانات Aurora-2. تتضمن قاعدة البيانات كلًا من الوضعين النظيف ومتعدد الشروط، وتشمل أربعة سيناريوهات للضوضاء: مترو الأنفاق، والثرثرة، والسيارة، وقاعة المعرض، يتم تقييم كل منها عند نسب إشارة إلى ضوضاء (SNRs) مختلفة، وحالة نظيفة لا تحتوي على ضوضاء، ويتم مقارنة النتائج بتلك الموجودة في قاعدة البيانات ASC-10 وقاعدة البيانات ESC-10. المشكلة التي تمت معالجتها في هذه الدراسة هي الحاجة إلى نماذج قوية للتعرف الآلي على الكلام التي تعمل بشكل جيد في كل من البيئات النظيفة والصاخبة (التي تحتوي على ضوضاء). والهدف من استخدام بنية CNN-LSTM هو تحسين أداء التعرف من خلال الجمع بين نقاط القوة في كل من النماذج CNNs وLSTMs، بدلاً من الاعتماد على CNNs أو LSTMs وحدها. تُظهِر النتائج التجريبية أن نموذج CNN-LSTM المدمج يحقق أداء تصنيف مرتفع جدا، في البيئات النظيفة على قاعدة البيانات Aurora2، حيث حقق نسبة 97.96% من الدقة، متجاوزًا نماذج CNN وLSTM عندما تأخذ فرديا، والتي حققت 97.21% و96.06% على التوالي. في الظروف الصاخبة، يتفوق النموذج الهجين أيضًا على النماذج المستقلة، بدقة 90.72%، مقارنة بـ 90.12% لـ CNN و86.12% لـ LSTM في النهاية، تشير هذه النتائج إلى أن نموذج CNN-LSTM أكثر فعالية في التعامل مع ظروف الضوضاء المختلفة وتحسين دقة التعرف على الكلام بشكل عام.
##plugins.themes.bootstrap3.article.details##
كيفية الاقتباس
جفالن., عدوج., خدارح., & سلوانيس. أ. (2024). دمج تقنية CNN-LSTM لتحسين التعرف على الكلام النظيف والصاخب. AL-Lisaniyyat, 30(2), 5-26. استرجع في من https://crstdla.dz/ojs/index.php/allj/article/view/732
القسم
Articles
المراجع
Alsayadi, H.A. et al. (2021). Non-diacritized Arabic speech recognition based on CNN-LSTM and attention-based models. Journal of Intelligent & Fuzzy Systems, 41(6), 6207-6219.
Daouad, M., Allah, F. A. & Dadi, E. W. (2023). An automatic speech recognition system for isolated Amazigh word using 1D & 2D CNN-LSTM architecture. International Journal of Speech Technology, 26(3), 775-787.
Dar M.A. & J. Pushparaj, J. (2024). Hybrid architecture cnn-blstm for automatic speech recognition. 3rd International Conference on Artificial Intelligence for Internet of Things (AIIoT). pp. 1–4.
Dat, T.T. et al. (2021). Convolutional recurrent neural network with attention for Vietnamese speech to text problem in the operating room. International Journal of Intelligent Information and Database Systems, 14(3), 294-314.
Demir, F., Turkoglu, M., Aslan, M. & Sengur, A. (2020). A new pyramidal concatenated CNN approach for environmental sound classification. Applied Acoustics, 170, 107520.
Djeffal, N. et al. (2023). Automatic speech recognition with BERT and CTC transformers: A review. In 2nd International Conference on Electronics, Energy and Measurement (IC2EM) (Vol. 1, pp. 1-8).
Djeffal, N. et al. (2023). Noise-robust speech recognition: A comparative analysis of LSTM and CNN approaches. In 2023 2nd International Conference on Electronics, Energy and Measurement (IC2EM) (Vol. 1, pp. 1-6). IEEE.
Greg V.H., Carlos M. & Gonzalo, N. (2020). A review on the long short- term memory model. Artificial Intelligence Review. Springer, 2020, pp. 5929–5955.
Gueriani, A., Kheddar, H. & Mazari, A. C. (2024). Enhancing IoT Security with CNN and LSTM-Based Intrusion Detection Systems. In 2024 6th International Conference on Pattern Analysis and Intelligent Systems (PAIS) (pp. 1-7). IEEE.
Essaid, B. et al. (2024). Artificial Intelligence for Cochlear Implants: Review of Strategies, Challenges, and Perspectives. IEEE Access.
Exter, M. & Meyer, B. T. (2016). DNN-Based Automatic Speech Recognition as a Model for Human Phoneme Perception. In INTERSPEECH (pp. 615-619).
Ghandoura, A., Hjabo, F. & Al Dakkak, O. (2021). Building and benchmarking an Arabic Speech Commands dataset for small-footprint keyword spotting. Engineering Applications of Artificial Intelligence, 102, 104267.
Habchi, Y. et al. (2023). Ai in thyroid cancer diagnosis: Techniques, trends, and future directions. Systems, 11(10), 519.
Hirsch, H. G. & Pearce, D. (2000). The Aurora experimental framework for the performance evaluation of speech recognition systems under noisy conditions. In ASR2000-Automatic speech recognition: challenges for the new Millenium ISCA tutorial and research workshop (ITRWH.
Karam, S. et al. (2023). Episodic memory based continual learning without catastrophic forgetting for environmental sound classification. Journal of Ambient Intelligence and Humanized Computing, 14(4), 4439-4449.
Kheddar, H. et al. (2023). Deep transfer learning for automatic speech recognition: Towards better generalization. Knowledge-Based Systems, 277, 110851.
Kheddar, H. et al. (2024). Deep learning for steganalysis of diverse data types: A review of methods, taxonomy, challenges and future directions. Neurocomputing, 127528.
Kheddar, H., Hemis, M. & Himeur, Y. (2024). Automatic speech recognition using advanced deep learning approaches: A survey. Information Fusion, 102422.
Li, J., Mohamed, A., Zweig, G. & Gong, Y. (2015). LSTM time and frequency recurrence for automatic speech recognition. In 2015 IEEE workshop on automatic speech recognition and understanding (ASRU) (pp. 187-191). IEEE.
Lichouri, M., Lounnas, K. & Bakri, A. (2023). Toward building another arabic voice command dataset for multiple speech processing tasks. In 2023 International Conference on Advances in Electronics, Control and Communication Systems (ICAECCS) (pp. 1-5). IEEE.
Mazari, A. C. & Kheddar, H. (2023). Deep learning-based analysis of Algerian dialect dataset targeted hate speech, offensive language and cyberbullying. International Journal of Computing and Digital Systems..
Naing, H.M.S., Hidayat, R., Hartanto, R. & Miyanaga, Y. (2020, November). A front-end technique for automatic noisy speech recognition. 23rd conference of the oriental COCOSDA international committee for the co-ordination and standardisation of speech databases and assessment techniques (O-COCOSDA) (pp. 49-54).
Passricha, V. & Aggarwal, R. K. (2019). A hybrid of deep CNN and bidirectional LSTM for automatic speechrecognition. Journal of Intelligent Systems, 29(1), 1261-1274.
Piczak, K. J. (2015). ESC: Dataset for environmental sound classification. In Proceedings of the 23rd ACM international conference on Multimedia (pp. 1015-1018).
Selouani, S. A. & Yacoub, M. S. (2018). Long short-term memory neural networks for artificial dialogue generation. In 2018 IEEE 42nd Annual Computer Software and Applications Conference (COMPSAC) (Vol. 1, pp. 761-768). IEEE.
Soe Naing, H.M. at al. (2020). Discrete Wavelet Denoising into MFCC for Noise Suppressive in Automatic Speech Recognition System. International Journal of Intelligent Engineering & Systems, 13(2).
Takazawa, S. K. et al. (2024). Explosion Detection Using Smartphones: Ensemble Learning with the Smartphone High-Explosive Audio Recordings Dataset and the ESC-50 Dataset. Sensors, 24(20), 6688.
Wu, Y., Zheng, B. & Zhao, Y. (2018). Dynamic gesture recognition based on LSTM-CNN. In 2018 Chinese Automation Congress (CAC) (pp. 2446-2450). IEEE.
Wang, W., Yang, X. & Yang, H. (2020). End-to-End low-resource speech recognition with a deep CNN-LSTM encoder. In 2020 IEEE 3rd International Conference on Information Communication and Signal Processing (ICICSP) (pp. 158-162). IEEE
Xie, J., Fang, J., Liu, C. & Li, X. (2020). Deep learning-based spectrum sensing in cognitive radio: A CNN-LSTM approach. IEEE Communications Letters, 24(10), 2196-2200.
Daouad, M., Allah, F. A. & Dadi, E. W. (2023). An automatic speech recognition system for isolated Amazigh word using 1D & 2D CNN-LSTM architecture. International Journal of Speech Technology, 26(3), 775-787.
Dar M.A. & J. Pushparaj, J. (2024). Hybrid architecture cnn-blstm for automatic speech recognition. 3rd International Conference on Artificial Intelligence for Internet of Things (AIIoT). pp. 1–4.
Dat, T.T. et al. (2021). Convolutional recurrent neural network with attention for Vietnamese speech to text problem in the operating room. International Journal of Intelligent Information and Database Systems, 14(3), 294-314.
Demir, F., Turkoglu, M., Aslan, M. & Sengur, A. (2020). A new pyramidal concatenated CNN approach for environmental sound classification. Applied Acoustics, 170, 107520.
Djeffal, N. et al. (2023). Automatic speech recognition with BERT and CTC transformers: A review. In 2nd International Conference on Electronics, Energy and Measurement (IC2EM) (Vol. 1, pp. 1-8).
Djeffal, N. et al. (2023). Noise-robust speech recognition: A comparative analysis of LSTM and CNN approaches. In 2023 2nd International Conference on Electronics, Energy and Measurement (IC2EM) (Vol. 1, pp. 1-6). IEEE.
Greg V.H., Carlos M. & Gonzalo, N. (2020). A review on the long short- term memory model. Artificial Intelligence Review. Springer, 2020, pp. 5929–5955.
Gueriani, A., Kheddar, H. & Mazari, A. C. (2024). Enhancing IoT Security with CNN and LSTM-Based Intrusion Detection Systems. In 2024 6th International Conference on Pattern Analysis and Intelligent Systems (PAIS) (pp. 1-7). IEEE.
Essaid, B. et al. (2024). Artificial Intelligence for Cochlear Implants: Review of Strategies, Challenges, and Perspectives. IEEE Access.
Exter, M. & Meyer, B. T. (2016). DNN-Based Automatic Speech Recognition as a Model for Human Phoneme Perception. In INTERSPEECH (pp. 615-619).
Ghandoura, A., Hjabo, F. & Al Dakkak, O. (2021). Building and benchmarking an Arabic Speech Commands dataset for small-footprint keyword spotting. Engineering Applications of Artificial Intelligence, 102, 104267.
Habchi, Y. et al. (2023). Ai in thyroid cancer diagnosis: Techniques, trends, and future directions. Systems, 11(10), 519.
Hirsch, H. G. & Pearce, D. (2000). The Aurora experimental framework for the performance evaluation of speech recognition systems under noisy conditions. In ASR2000-Automatic speech recognition: challenges for the new Millenium ISCA tutorial and research workshop (ITRWH.
Karam, S. et al. (2023). Episodic memory based continual learning without catastrophic forgetting for environmental sound classification. Journal of Ambient Intelligence and Humanized Computing, 14(4), 4439-4449.
Kheddar, H. et al. (2023). Deep transfer learning for automatic speech recognition: Towards better generalization. Knowledge-Based Systems, 277, 110851.
Kheddar, H. et al. (2024). Deep learning for steganalysis of diverse data types: A review of methods, taxonomy, challenges and future directions. Neurocomputing, 127528.
Kheddar, H., Hemis, M. & Himeur, Y. (2024). Automatic speech recognition using advanced deep learning approaches: A survey. Information Fusion, 102422.
Li, J., Mohamed, A., Zweig, G. & Gong, Y. (2015). LSTM time and frequency recurrence for automatic speech recognition. In 2015 IEEE workshop on automatic speech recognition and understanding (ASRU) (pp. 187-191). IEEE.
Lichouri, M., Lounnas, K. & Bakri, A. (2023). Toward building another arabic voice command dataset for multiple speech processing tasks. In 2023 International Conference on Advances in Electronics, Control and Communication Systems (ICAECCS) (pp. 1-5). IEEE.
Mazari, A. C. & Kheddar, H. (2023). Deep learning-based analysis of Algerian dialect dataset targeted hate speech, offensive language and cyberbullying. International Journal of Computing and Digital Systems..
Naing, H.M.S., Hidayat, R., Hartanto, R. & Miyanaga, Y. (2020, November). A front-end technique for automatic noisy speech recognition. 23rd conference of the oriental COCOSDA international committee for the co-ordination and standardisation of speech databases and assessment techniques (O-COCOSDA) (pp. 49-54).
Passricha, V. & Aggarwal, R. K. (2019). A hybrid of deep CNN and bidirectional LSTM for automatic speechrecognition. Journal of Intelligent Systems, 29(1), 1261-1274.
Piczak, K. J. (2015). ESC: Dataset for environmental sound classification. In Proceedings of the 23rd ACM international conference on Multimedia (pp. 1015-1018).
Selouani, S. A. & Yacoub, M. S. (2018). Long short-term memory neural networks for artificial dialogue generation. In 2018 IEEE 42nd Annual Computer Software and Applications Conference (COMPSAC) (Vol. 1, pp. 761-768). IEEE.
Soe Naing, H.M. at al. (2020). Discrete Wavelet Denoising into MFCC for Noise Suppressive in Automatic Speech Recognition System. International Journal of Intelligent Engineering & Systems, 13(2).
Takazawa, S. K. et al. (2024). Explosion Detection Using Smartphones: Ensemble Learning with the Smartphone High-Explosive Audio Recordings Dataset and the ESC-50 Dataset. Sensors, 24(20), 6688.
Wu, Y., Zheng, B. & Zhao, Y. (2018). Dynamic gesture recognition based on LSTM-CNN. In 2018 Chinese Automation Congress (CAC) (pp. 2446-2450). IEEE.
Wang, W., Yang, X. & Yang, H. (2020). End-to-End low-resource speech recognition with a deep CNN-LSTM encoder. In 2020 IEEE 3rd International Conference on Information Communication and Signal Processing (ICICSP) (pp. 158-162). IEEE
Xie, J., Fang, J., Liu, C. & Li, X. (2020). Deep learning-based spectrum sensing in cognitive radio: A CNN-LSTM approach. IEEE Communications Letters, 24(10), 2196-2200.