Enhanced secures de duplication for big data storage with integrity preservation in cloud

Abstract

Cloud computing and the remarkable increase in the volume of data that are pooled into cloud storage devices from various sources and applications have paved the way for the emergence of big data. The large volume of data comes with redundancy, need de-duplication and integrity delegation. De-duplication along with data integrity frees up the space in the cloud server and ensures the security of the stored data. The proposed zero-knowledge cross-user SBDedupe technique files into chunks and a hash value is generated for each chunk using cuckoo hashing which reduces the time and bandwidth. The core of the proposed method is made up of three parts. In the first part, to identify the duplicates, an enhanced cuckoo hashing is employed, which is significantly faster than the traditional hashing algorithms. Following this, the de-duped data hash index is transmitted and checked for de-duplication in the Cloud Service Provider (CSP) for which the missing hash values of the corresponding data chunk alone will be sent to the CSP, and the data lost in the process is recovered using Reed-Solomon (RS) mechanism. The proposed key exchange algorithm inherits the security enhancement of the Advanced Encryption Standard (AES). The integrity check for zero-knowledge cross-user SBDedupe of big data in cloud storage is obtained by the integration of Message Locked Encryption - Third Party Auditor key. The proposed framework uses less computation power and frees up storage space up to 86.77 % which is higher than other state-of-the-art deduplication algorithms. The upload and download response time for file encoding is stable even when the file size is more prodigious. Subsequently, the proposed zero-Knowledge cross-user data de-dupe mechanism is stable despite the variation in file size. newline