DDFP: Duplicate detection and fragment placement in deduplication system for security and storage space

Due to increasing volume of data in information technology, saving storage space and providing security to data has acquired more attention and popularity. In data processing and data mining, duplicates can effect severely. Data deduplication is a technique that eliminates duplicate data and store only one copy, promoting single instance storage. The main challenges are to identify maximum duplicate segment and selecting the storage nodes for distributing fragments of files. In this paper we proposed, Duplicate Detection and Fragment Placement (DDFP) a deduplication system that effectively eliminates duplicate data and fragments placement that allocates unique instances of a data file on storage nodes. For repeated data, reference pointer is used and only unique data is stored on the storage node. This increases the percentage of duplicate data detection. A fragment placement algorithm is used for placing fragments on different storage nodes. To select nodes T-coloring is used, Set T is used, which restricts the nodes that are at distance T from one another. DDFP considerably achieves duplicate elimination and obtain the high level of security on data fragments by storing fragments of the data file using T-coloring. This Selects the nodes that are not adjacent which prevent unauthorized access to data from other users.