This week I had question from our field concerning a customer and LUN/device maximums which I think is useful to share. The customer was running vSphere 6.5 which supports 512 LUNs per ESXi host (per target/array) but they were unable to see any beyond 255. Now before 6.5, I frequently heard from customers who hit the magic 255 LUN ID, or more commonly couldn’t even reach LUN ID 255 and were confused. Now some customers did not know there was the 256 device maximum (don’t forget LUN ID 0) , but others were hitting the other limit – paths. Up to 6.5, ESXi only supported 1024 paths. That means you could have 4 paths for each device if you maxed it all – 256 x 4 = 1024. Many customers, however, use 8 paths so when they hit 128 devices they are done. The only solution to that is to use less paths, and frankly most of the time 8 paths are unnecessary anyway. There certainly are environments where the Dell EMC performance specialists determine 8 is the right number, but generally you can drive enough through 4 paths across 4 ports on the array. Now don’t go crazy changing your configuration if you use 8 paths. Always check with your Dell EMC team first.
So when the field contacted me, the customer had helpfully included a blog link they had used to validate their reasoning that despite 6.5 supporting 512, the VMAX would only support 256. My friend Cody has actually written it and indeed the conclusion was that you needed to wait until vSphere 6.7 to exceed 256. Now I had never had an issue presenting more than 256 devices with vSphere 6.5, but I do what I always do, sanity test. So I quickly ran one and let the customer know I did not find a problem by providing this:
I also read through Cody’s post where he does a nice job of explaining LUN addressing and the difference between peripheral and flat. He explains that beyond the 255 LUN ID flat addressing is used and VMware doesn’t support it until vSphere 6.7, hence the conclusion that even in 6.5, 256 devices is the maximum. True enough if you are running a Pure array. Cody does work for Pure after all (the orange gives it away). We use peripheral addressing with the VMAX/PowerMax and hence we can go all they way up to 512 devices just as VMware supports. Cody told me he had actually planned on updating the post to point this out about peripheral but hadn’t gotten around to it (he has now). And there is no point in me talking about addressing further when you can read his good stuff.
OK, then what was going on with our customer? Paths – the other pitfall. vSphere 6.5 supports a hefty 2048, but if you are using 8 paths, you can do the math and see 256 devices is your maximum.