Before I discuss data reduction on the PowerMax, I will acknowledge that this can be a confusing topic or process in general. A couple of our field guys, Dan McKenney and Ken Gotsch, get frequent questions in the field as they work with data reduction all the time in VMware, so I offered to write a bit about it. My post here is only meant to shine a little light on the storage group compression ratio in VMware environments as it tends to be the focus of most customer questions. My examples and experience may or may not be similar to yours because every customer’s data is unique and thus responds to compression differently. Yes, there are some generalizations but I won’t go so far as to call them tenets or axioms. If you’re looking for the nuts and bolts of PowerMax Data Reduction Ratio (DRR) in totality, it can be found in the white paper Dell EMC PowerMax: Data Reduction. It includes detailed explanations and formulas.
Data Reduction
On our PowerMax arrays, we have data reduction which is a combination of inline compression and inline deduplication (VMAX All Flash has compression only). The inline compression is controlled by an Adaptive Compression Engine (ACE) which uses both hardware and software compression. Deduplication does not have an equivalent engine, but it does use the same hardware to perform the compression. Deduplication utilizes hash ids and hash tables to determine what data is redundant (once compression is done). At a high level, a track comes in and is assigned a hash id. That hash id is compared against a hash table with stored ids. If it already exists in the table, the PowerMax doesn’t have to store the data, rather it just sets a pointer (or uses the existing one) to the source. If the hash id does not exist in the table, it is stored there, the data kept, and is available for future comparisons.
Enabling Data Reduction
Data reduction is enabled at the storage group level. When creating a single group, the box is checked by default.
If you are using child storage groups, each one can have a different data reduction setting. Again, by default it will be enabled on all groups, but if you select the pencil next to the group in step 1, you’ll see in step 2 where you can uncheck the box, and then the final result in step 3 is only one group has data reduction.
In general Dell EMC recommends using data reduction for all storage groups unless it is known that the devices in the storage group will house data that is not favorable to compression or deduplication. For example, a device that will be used as a datastore for storing a content library of ISOs, zips, etc. is probably a good candidate for disabling data reduction. Any application that uses software encryption would be another.
Ratios/Efficiencies
To view the compression data reduction each storage group is receiving (deduplication is not included in this calculation), use Unisphere for PowerMax where there are two relevant columns, Compression Ratio and Unreducible (GB). Below is the high-level storage group view.
To understand how the Compression Ratio was achieved, however, it is necessary to run the Storage Group Demand Report. The report has the columns necessary to calculate the Compression Ratio, which are Allocated (GB) divided by Used (GB). The group I have highlighted below, drr_testing_sg, has 950.19 GB allocated and 686.40 GB used, for a Compression Ratio of 1.38:1 (rounded below to 1.4). This group will be used below in our example.
The Unreducible amount of 5.60 GB is not additional data, rather it is the portion of the Used data that has no compression. We call this amount out separately as it can be a good indicator as to why a Compression Ratio is low, i.e. a large value in proportion to the Used will invariably mean a lower Compression Ratio. I should note the unreducible data refers to data that is received on the array in a state that is already compressed/encrypted and cannot be further manipulated. This wouldn’t be zip files, for example, since there is always better compression, rather something like an audio file which is a compressed file by design.
PowerMax Data Reduction Ratio
If you would like addition efficiency ratios, which will include deduplication, these are available at the SRP level. To understand all these numbers, please see the previously mentioned white paper.
Before discussing the drr_testing_sg storage group in particular, I want to mention a number of things that will have an impact on what your particular compression ratio is.
What Impacts Compression?
Most VMware environments see about 2/2.5:1 Compression Ratio at the storage group level. Obviously there are many factors that influence that number so I’m going to talk about some of them and then use my storage group as an example.
VMware Memory Compression Cache
VMware by default utilizes something called a memory compression cache to improve performance when memory is overcommitted on the ESXi host. Overcommitting memory is exactly what it sounds like, the total allocated memory of your VMs is more than the physical memory. Now you can overcommit and never have an issue because you have so many idle VMs versus active ones, but if you get in a state where you’ve hit your physical RAM limits, ESXi will compress the virtual pages and store them. When a page needs to be swapped, VMware will try to compress it. Accessing this compressed memory is faster than accessing swapped memory to disk. Memory Compression Cache is enabled by default and is set to 10%. You can see the setting Mem.MemZipMaxPct below in Advanced System Settings. It can be increased or reduced to zero.
I would not recommend turning this off as it can be very helpful if you overcommit the memory. The flip side, however, is that if the memory is compressed by ESXi, it likely cannot be compressed again on the array. It may in fact be considered unreducible. Since the amount is limited to 10% of the VM memory and is only a temporary occurrence, this should not be a significant factor in impacting the Compression Ratio. You can check if your VM is actively using this cache by viewing the Utilization in the vSphere Client:
Windows Memory Compression
Beginning with Windows 10 and Windows Server 2016, Microsoft offers memory compression. Compressing memory offers a more efficient use of the RAM, and generally helps performance. It is enabled by default on Windows 10 but disabled by default on Windows Server. A simple PowerShell command can turn it on for the Server.
Once activated, it does not mean Windows compresses all the memory at once; rather it uses it for a portion of the memory. You can see how much by using the performance monitor and viewing the number in parentheses under the In use memory. In this example I have 1.8 GB of memory in use, 10 MB of which is compressed.
For customers that use Windows GuestOS and activate this feature, any compressed memory, like that for the VMware memory compression cache, likely will not benefit from the array compression. In my example it is a negligible amount, but in very large VMs, say big SQL Server databases, it may be more significant.
Eager Zeroed Thick Disks (EZT)
EZT questions come up in the field sometimes when discussing DRR. Generally we don’t recommend EZT vmdks unless an application or VMware feature (Fault Tolerance) requires it. Our PowerMax arrays don’t actually write zeroes when using this type of vmdk because of VAAI. If you want the detail, I’ve got a post for that 🙂 But when it comes to DRR there can be confusion around whether an EZT vmdk causes the data reduction ratios to be skewed. They do not. As we do not allocate space, the array has no idea we are using EZT vmdks. For the sake of argument though, let’s say VAAI is disabled and VMware has to write the zeroes. If DRR is enabled on a storage group, the PowerMax simply strips the zeroes before they are written. So again, no allocations are made, no impact to the ratios.
Application Considerations
Customer applications are probably the biggest factor that can impact data reduction. For example, if your application has the ability to compress data before storing it, it is unlikely the PowerMax can squeeze out much else. As I mentioned, it is possible for the PowerMax to compress a bit more out of a zip file, but still it won’t be much. Oracle Compression is a good example here, too. And though we aren’t talking about deduplication, similarly, if the application encrypts data before storing it, the PowerMax won’t be able to deduplicate. Again, Oracle Encryption is like this. Now I am in no way discouraging the use of these application technologies. This post isn’t about that. I’m just trying to help our customers understand why they might have particular ratios.
Ultimately you may decide that some of these features are really useful on the application side. For example, in an Oracle database I may want to encrypt personal data (e.g. HR, payroll, etc.) but not the rest. Yes that personal data will not be deduplicated, but it is a very small part of the whole database. Sybase compression? Not my area but the TCs tell me it is essential for performance so it wins over the array (and its devices can be in an SG without data reduction). There are subtleties here in using both application and array technologies to your benefit. It’s worth the conversation with your Dell representatives. Also if you aren’t sure whether your application may compress or encrypt, check with the vendor.
Compression Example
So let’s return to my storage group. I’ve removed the snapshot numbers from the demand report below since I only want to concentrate on the Compression Ratio. As I noted, the ratio is allocated (950.19) divided by the used (686.40). This results in a ratio of ~1.4:1, not so great. I also have a small amount of unreducible data ~5.6 GB included in that used, though not enough to explain the less than desirable ratio.
As this ratio is somewhat lower than what we might expect (2:1 on the low side) in a VMware environment, we will try to determine what is impacting that. This SG contains a single device with a datastore, DRR_355_TESTING, and a number of VMs.
I know that I am not running any applications that would significantly impact the ratio, however I am using memory compression on the GuestOS as these are all Windows so there is some loss there. In addition, being VMware there is the 10% memory cache which could play a role, though I see I am not overcommitted on my ESXi hosts. What I do know, is that the VM drr6 contains most of the software I use in my lab so there are lots of zips, isos, and ovas, none of which compress very well. So to see if this might be enough to skew my ratio I’m going to generate a size for those files.
I’m going to use PowerShell to search recursively through my two Windows drives on that VM and grab all the zip, iso, and ova extension files and add up their size. If I had jpeg or audio I might include those but I know I don’t. As an aside those files would likely show up as unreducible anyway.
Adding up those numbers I have about 424 GB of files which have poor compression, and that’s just in this single VM. So I’m just going to Storage vMotion this VM to a new datastore. This will trigger VMware to issue automatic UNMAP against the datastore and reclaim the storage. When it completes, here is my new storage group demand report with a more average Compression Ratio of 1.9:1, and much closer to what I expect.
So here is a situation where perhaps this VM joins similar ones in a storage group that does not have data reduction enabled. It is not necessary, however. This example was simply to demonstrate how the type of data being stored in a VM can influence the compression ratio. Knowing this, you may still want the VM where it is, but are less concerned about the ratio as you know why it is that value.
Wrap up
I hope this was helpful in understanding how the compression ratio is influenced by the type of data that is stored on your VMs as well as some compression features of VMware and Windows. DRR on the PowerMax can be a confusing topic, but by focusing on the type of data you are storing, you are more likely to understand the reasons for the ratios.
Dell PowerFlex and PowerMax with VMware 
Very interesting info Drew. Well done.