Managing cloud-generated logs using big data technologies

Lemoudden, Mouad; El Ouahidi, Bouabid

Abstract

Cloud computing is a fast-growing paradigm that has forcefully emerged and established itself in the next generation of IT industry and business, performing massive-scale and complex computing. As cloud technology adoption continues to progress, massive growth in the scale of data generated through cloud computing has been observed. A large part of that data has been machine-generated log data; it is generated at every layer and component within the cloud information technology ecosystem that spans a range of IT functions from storage and computation to networking and application services. Log files are essential for enterprise-level monitoring, troubleshooting, security, debugging, compliance, etc. We argue that cloud computing is in need of a new methodology for recording, managing, storing and analyzing the log data that is generated by the cloud infrastructure, based on big data technologies, allowing for the access to a wealth of knowledge that can contribute to its own advancement. Our paper discusses and expands existing standard log management solutions to design new methods of exploiting the unstructured log data in order to gain unprecedented insights.

Citation

Lemoudden, M., & El Ouahidi, B. (2015). Managing cloud-generated logs using big data technologies. In 2015 International Conference on Wireless Networks and Mobile Communications (WINCOM). https://doi.org/10.1109/wincom.2015.7381334

• Conference Name: 2015 International Conference on Wireless Networks and Mobile Communications (WINCOM)
• Conference Location: Marrakech, Morocco
• Start Date: Oct 20, 2015
• End Date: Oct 23, 2015
• Online Publication Date: Jan 14, 2016
• Publication Date: 2015-10
• Deposit Date: Feb 28, 2023
• Publisher: Institute of Electrical and Electronics Engineers
• Book Title: 2015 International Conference on Wireless Networks and Mobile Communications (WINCOM)
• DOI: https://doi.org/10.1109/wincom.2015.7381334
• Keywords: cloud computing, logs, big data, text analytics, logging, unstructured data, log management