Comparison Of Common Mathematical Techniques Used In The Calculation Of File Entropy

Davies, Simon R.; Macfarlane, Richard

Abstract

The research described in this paper focuses on the use of mathematical techniques to identify high entropy encrypted files generated during the execution of ransomware. A common approach used by many ransomware detection techniques is to monitor file system activity and attempt to identify encrypted files being written to disk, often using the file's entropy as an indicator of encryption. However, often in the description of these techniques, little or no discussion is made as to why a particular entropy calculation technique is used or any justification given as to why one technique was selected over the alternatives. The paper compares the suitability for five of the most common mathematical techniques currently being used for entropy calculation: Chi-Square (χ2) , Shannon Entropy, Mean Average, Monte Carlo estimation, and Serial Correlation. During the testing, the five separate entropy values were calculated for each of the 245,000 target files - resulting in nearly 1.2 million separate calculations. The overall accuracy of each of the individual test's ability to differentiate between high entropy files encrypted using ransomware and other file types is then evaluated. Each test is compared using this metric in an attempt to identify the entropy method most suited for encrypted file identification. The hypothesis being that there is a fundamental difference between different entropy methods and that the best methods can be used to better detect random ware encrypted files

Citation

Davies, S. R., & Macfarlane, R. (2022, November). Comparison Of Common Mathematical Techniques Used In The Calculation Of File Entropy. Presented at 2022 International Conference on Electrical, Computer, Communications and Mechatronics Engineering (ICECCME), Maldives

• Presentation Conference Type: Conference Paper (Published)
• Conference Name: 2022 International Conference on Electrical, Computer, Communications and Mechatronics Engineering (ICECCME)
• Start Date: Nov 16, 2022
• End Date: Nov 18, 2022
• Acceptance Date: Nov 16, 2022
• Online Publication Date: Dec 30, 2022
• Publication Date: 2022
• Deposit Date: Jan 13, 2023
• Publisher: Institute of Electrical and Electronics Engineers
• Book Title: 2022 International Conference on Electrical, Computer, Communications and Mechatronics Engineering (ICECCME)
• DOI: https://doi.org/10.1109/iceccme55909.2022.9988132