Windows Server\u2019s Data Deduplication feature can make it seem as though storage figures are behaving unexpectedly. Adding several copies of a document may use less space than expected, while deleting a file\u2014and even emptying the Recycle Bin\u2014may not immediately free up capacity.<\/p>\n
The reason is that deduplication does not simply compare whole files. It breaks data into chunks, stores only unique chunks, and reconstructs files when they are accessed. Two details explain much of the delay: files may not be optimized until they are three days old, and space from deleted files may not be reclaimed until the weekly cleanup job runs.<\/p>\n
Microsoft describes Data Deduplication as following two principles: it should not interfere with disk writes, and it should not change how users or applications access files.<\/p>\n
To avoid slowing down writes, Windows first saves files in their ordinary, unoptimized form. A later optimization job processes eligible data. This means the feature works in the background, but storage savings are not necessarily visible as soon as files are copied.<\/p>\n
Deduplication identifies repeated patterns across files, divides data into chunks, and stores unique chunks in a chunk store. This can save space even when files are not identical\u2014for example, when several documents share most of their content but have different cover pages.<\/p>\n
The original file stream is replaced with a reparse point, while details such as the file name and creation date remain. When a file is opened, the file system directs the request to the Data Deduplication filter, Edits to an optimized file are initially written in unoptimized form. The next optimization job processes those changes, so files that are frequently updated may have portions that are not yet deduplicated.<\/p>\n The chunk store is kept in container files under the volume\u2019s Windows Server offers three usage types: Default<\/strong>, Hyper-V<\/strong>, and Backup<\/strong>.<\/p>\n The three-day delay is therefore a policy setting for particular workloads, not an inherent requirement of deduplication. Backup workloads can be optimized from the day files are added.<\/p>\n The optimization job checks according to its schedule, which defaults to once an hour. The hourly schedule determines how often the job runs; the file-age policy determines whether a particular file is eligible when it runs.<\/p>\n Data Deduplication uses four jobs:<\/p>\nDedup.sys<\/code>, which retrieves the necessary chunks and assembles the file for reading.<\/p>\nSystem Volume Information<\/code> folder. Microsoft warns against modifying it manually: doing so can corrupt or destroy data.<\/p>\nWhen optimization takes effect<\/h2>\n
\n
Why deleting files may not free space right away<\/h2>\n