Sabrent Rocket M.2 2TB PCIe Gen4 QLC performance compared to 1TB PCIe Gen4 TLC

noko

Supreme [H]ardness
Joined
Apr 14, 2010
Messages
7,262
Just picked up a Sabrent 2TB QLC drive for $280 on sale. Almost picked up two for Raid, which would probably make the PS5 not look so beastly. Then reality hit, I would probably never notice the difference other than a benchmark, unless someone here can explain why it would be a good option.

Basically the new M.2 SSD was placed in the two slot MSI adapter that came with the MSI TRX40 Pro WiFi motherboard, which has a TR 3960x, placed in an PCIe 16x slot wired for 8x Gen4. 1st image is M.2 drive being installed in adaptor, 2nd and 3rd the card on the motherboard. The adaptor can hold up to two PCIe M.2 SSDs.


M2inMSIadaptor.jpgCardinSystem.jpgCardinSystemTop.jpg
Crystal Diskmark scores, first is from the 2TB drive with Quad Level Cells, 2nd is from the 1TB Triple Layer Cells

Sabrient2TBpcie4_QLC.png
Sabrient1TBpcie4_TLC.png

As expected the TLC Gen4 drive is faster than the QLC version. Still for less than a $100, an extra TB is obtain. Since the drive is empty, I will see how making it a compressed drive does with performance.
 
Last edited:
Made the 2TB Gen4 SSD compressed and here are the results:

Compressed2TBQLCdrive.png

Not much change in the end, thought maybe with compressed data which would be less on the drive and to read from would be faster.
 
Those are some stupid speeds. What is thew cache size on those drives? once you fill that wont performance drop a lot?
 
Those are some stupid speeds. What is thew cache size on those drives? once you fill that wont performance drop a lot?
Don't know, probably when you get past 80% capacity, maybe less. As for real world difference, I just don't feel it :). Anyways the 1tb drive is the system drive and at 29% capacity while the 2tb is basically 0% full.
 
Hey, just FYI I'm going to be reviewing this exact drive (the Rocket Q4 2TB) over on STH somewhat soon. I have the drive in my possession right now, and I've already benchmarked it. However, I've been notified there is a potential firmware issue though and , and that the firmware problem causes some performance problems. I don't have any of the details yet.

In general though, even with the drive as it exists now, if sequential read speed is your goal this drive is pretty damn speedy and easily wipes the floor with the original Rocket Q 2TB drive running on PCIe 3.0.

One comment - this drive needs cooling a bit. It runs pretty warm, so it needs some kind of airflow at the minimum or preferably a heatsink. Under light load it probably wouldn't be an issue, but under heavy strain it heateded up well above 70C.
 
What controller is this?

Did you try benching with the DRAM cache completely filled up/used?
 
It's a Phison E16 controller. The DRAM cache is small ( it is on all SSDs) but the SLC cache is variable based on remaining capacity on the drive. I run my benchmark suite with the drives at ~65% used capacity and in a 'dirty' state as far as TRIM is concerned. I fill the drive up to ~85%, then delete enough data to get to 65%, and then run benchmarks without initiating a trim inbetween. My goal is to get the drive to a healthy kind of "middle-aged" capacity point, since I feel like this is where lots of drives end up. Let's face it, if your drive was 99% full you would be replacing it, not benchmarking it.
 
Hey, just FYI I'm going to be reviewing this exact drive (the Rocket Q4 2TB) over on STH somewhat soon. I have the drive in my possession right now, and I've already benchmarked it. However, I've been notified there is a potential firmware issue though and , and that the firmware problem causes some performance problems. I don't have any of the details yet.

In general though, even with the drive as it exists now, if sequential read speed is your goal this drive is pretty damn speedy and easily wipes the floor with the original Rocket Q 2TB drive running on PCIe 3.0.

One comment - this drive needs cooling a bit. It runs pretty warm, so it needs some kind of airflow at the minimum or preferably a heatsink. Under light load it ttttyyyyyyyyprobably wouldn't be an issue, but under heavy strain it heateded up well above 70C.
Thanks for the info. Look forward to your review.

Temperture was main reason why I stuck the 2TB drive in the adapter which has a fan vice the motherboard. The 1TB drive is on the motherboard and the gpu sits right on top of it. Temperture goes over 85c when being stressed constantly with Crystal Diskmark. The 2TB drive never goes over 50c with the adapter with fan and alot of metal mass.

Transferred 100gb from the 1TB Sabrent Gene4 TLC, to the 2TB Sabrent Gen4 QLC. Microsoft Flight Simulator Cache directory. Highest transfer rate during the whole transfer was briefly 1.1gb/sec most of the time under 300mb/sec. Not even close to the benchmarks. Wonder if Windows file manager is causing the limited speeds with single thread and low Q's?
 
Wonder if Windows file manager is causing the limited speeds with single thread and low Q's?
Almost certainly if the flight sim cache directory is a bunch of smaller files rather than one large one.
 
Yes. But you've already run CrystalDiskMark which effectively does the same thing, but bypasses any of the 'windows stuff' that might get in the way.
 
Yes. But you've already run CrystalDiskMark which effectively does the same thing, but bypasses any of the 'windows stuff' that might get in the way.
ok, just trying to wrap my head around real world benefit in Gen4 SSDs if the OS, programs really do not take advantage of the increase bandwidth. What applications, workloads will see a benefi?

Will run 100gb zip file run for transfer rates and temperatures.
 
Honestly? Right now, unless you have a specialized need for the increased speed, there is almost zero reason to upgrade to even regular PCIe 3.0 NVMe SSDs over their SATA predecessors, let alone move to a PCIe 4.0 SSD. If all you do is play games, the "speed" of the SATA drive doesn't currently matter.

The reason for this is that, right now, things like level load times are primarily bottlenecked by things other than the storage subsystem. Textures need decompression, shaders need compilation, network data needs downloading, etc. The actual amount of time spent waiting on the SSD is pretty minimal compared to those things, in most cases. Additionally, right now on the PC games are still designed with the lowest common denominator - a mechanical hard drive - in mind.

There *are* workloads that can stress out a SSD - high bitrate audio/visual data streaming/importing as an example off the top of my head - but there aren't currently any "common man" type examples.

That might change in the future, with games designed around the XSX and PS5, both of which are shipping with very fast SSDs and games are going to be designed around their ability to stream data very quickly from their storage subsystems. On the other hand, ports to PC will likely still take into account the idea that *lots* of PCs have slower storage subsystems and adapt accordingly. One huge advantage to PCs in this space is memory capacity - right now most new gaming PCs would come equipped with 16/32GB of main system memory, and GPUs come with 8GB or more of dedicated GPU memory. In contrast, both the new XSX and PS5 have to make do with 16GB of memory shared between the GPU and main system. PCs can have a longer up-front loading time and simply hold more level data in memory and have less need of high bandwidth streaming as a result.

The moral of the story is, right now if you just want to play games, buy the biggest SSD you can and don't much worry about the interface speed. That *could* change in the future, but for now having a SSD means you are tall enough to ride all the rides.
 
Honestly? Right now, unless you have a specialized need for the increased speed, there is almost zero reason to upgrade to even regular PCIe 3.0 NVMe SSDs over their SATA predecessors, let alone move to a PCIe 4.0 SSD. If all you do is play games, the "speed" of the SATA drive doesn't currently matter.

The reason for this is that, right now, things like level load times are primarily bottlenecked by things other than the storage subsystem. Textures need decompression, shaders need compilation, network data needs downloading, etc. The actual amount of time spent waiting on the SSD is pretty minimal compared to those things, in most cases. Additionally, right now on the PC games are still designed with the lowest common denominator - a mechanical hard drive - in mind.

There *are* workloads that can stress out a SSD - high bitrate audio/visual data streaming/importing as an example off the top of my head - but there aren't currently any "common man" type examples.

That might change in the future, with games designed around the XSX and PS5, both of which are shipping with very fast SSDs and games are going to be designed around their ability to stream data very quickly from their storage subsystems. On the other hand, ports to PC will likely still take into account the idea that *lots* of PCs have slower storage subsystems and adapt accordingly. One huge advantage to PCs in this space is memory capacity - right now most new gaming PCs would come equipped with 16/32GB of main system memory, and GPUs come with 8GB or more of dedicated GPU memory. In contrast, both the new XSX and PS5 have to make do with 16GB of memory shared between the GPU and main system. PCs can have a longer up-front loading time and simply hold more level data in memory and have less need of high bandwidth streaming as a result.

The moral of the story is, right now if you just want to play games, buy the biggest SSD you can and don't much worry about the interface speed. That *could* change in the future, but for now having a SSD means you are tall enough to ride all the rides.
Sounds pretty much right on there.

7Zipped the folder which was 100gb, which the 7Z file was close to 80gb. Much better transfer speed or write speed to the 2TB Sabrent Rocket Gen4 drive. 2.15gb/s. Something in the end a good PCIe 3 drive can maybe do even faster.

80gbZip.png
Temperature barely budged on the Sabrent 2TB Gen 4 during the copy paste action, did not take long to complete. The CrystalDiskInfo graphs are virtually worthless as a note, the smallest time span is 1 min. Looks like the Cache did fine for this task of around 80gb being transferred to it.

So raiding two Gen4 drives would be even more pointless than spending extra money on a Gen4 drive. Now this drive did come rather cheap for the above the needed performance. I will probably put another 2TB drive in the MSI m.2 XPANDER-Z GEN4 adaptor in the near future and not raid it since I think that would be pointless at this time.
 
Last edited:
Haha. Well when I'm done with my drive, I'll hit you up and see if you want to buy it so you can have a matched pair :p

FYI, I can explain the cache behavior on these drives.

NAND flash is made up of cells, and QLC stores 4 bits per each cell, where TLC stores 3 bits and MLC stores 2, SLC stores 1. The reason QLC has lower endurance and slower write speed is because if you need to change the contents of a 4-bit cell, you have to write the whole 4-bits, even if the amount of data that is changing is just 1 of the 4 bits. This isn't a problem when the drive is empty and you're writing big contiguous files for the first time, but later in life as you're changing existing data a small amount at a time, the rewriting operation not only lowers durability by causing the drive to absorb extra bits written (called write amplification) but flat out takes longer because of the whole rewriting data operation. The same thing happens on TLC drives, and even MLC drives, but to correspondingly lower degrees because fewer bits share a cell.

To compensate for this, the controller on drives like this Sabrent form what is called a pseudo-SLC cache. Basically, up to 25% of unused capacity of the drive is 'borrowed' by the controller and it *pretends* it is SLC. It does this by only writing 1 bit into each 4-bit cell, thus the operation to rewrite that cell is easy since there is only 1 bit in there it can just be overwritten. So on a 2TB drive, up to 500GB of the raw NAND can be borrowed for this pseudo-SLC operation, but since you're only storing 1 bit per cell instead of 4, your effective SLC capacity is 1/4 of the 500GB or 125GB. When the drive accepts an incoming write operation, the controller actually puts the data into this pseudo-SLC cache (where it can write quickly) and then during idle time it shuffles it over to the normal QLC. This avoids the pitfalls of QLC for almost all casual use. This is while the drive is *empty* though; as you fill up the drive, the 25% of unused capacity gets smaller and smaller, and can lead to somewhat catastrophic write performance falloff if that cache gets too small for day-to-day operations, which can happen if the drive is 99% full or something.

TLC drives do the same thing, their write penalty is simply smaller than QLC since it's 3 per cell instead of 4. MLC is 2 bits per cell and has a correspondingly lower write penalty, and true SLC has no write penalty, but essentially zero consumer SSDs are native SLC. TLC is kind of the performance 'sweet spot' where the penalty of writing direct to TLC isn't so bad, and the efficiency of offering a pseudo-SLC cache isn't as bad either at 1/3 capacity instead of 1/4.

The onboard DRAM cache on SSDs is mostly not used for cacheing incoming data writes, or I should say that isn't its most important job. it is instead used to cache an address mapping table. This table is somewhat analogous to a table of contents, it tells the flash drive where each of the bits are. DRAMless SSDs either have to make do without that table, which impacts performance, or some NVMe drives can use a protocol called HMB (host memory buffer) to borrow system memory to hold that table.
 
Haha. Well when I'm done with my drive, I'll hit you up and see if you want to buy it so you can have a matched pair :p

FYI, I can explain the cache behavior on these drives.

NAND flash is made up of cells, and QLC stores 4 bits per each cell, where TLC stores 3 bits and MLC stores 2, SLC stores 1. The reason QLC has lower endurance and slower write speed is because if you need to change the contents of a 4-bit cell, you have to write the whole 4-bits, even if the amount of data that is changing is just 1 of the 4 bits. This isn't a problem when the drive is empty and you're writing big contiguous files for the first time, but later in life as you're changing existing data a small amount at a time, the rewriting operation not only lowers durability by causing the drive to absorb extra bits written (called write amplification) but flat out takes longer because of the whole rewriting data operation. The same thing happens on TLC drives, and even MLC drives, but to correspondingly lower degrees because fewer bits share a cell.

To compensate for this, the controller on drives like this Sabrent form what is called a pseudo-SLC cache. Basically, up to 25% of unused capacity of the drive is 'borrowed' by the controller and it *pretends* it is SLC. It does this by only writing 1 bit into each 4-bit cell, thus the operation to rewrite that cell is easy since there is only 1 bit in there it can just be overwritten. So on a 2TB drive, up to 500GB of the raw NAND can be borrowed for this pseudo-SLC operation, but since you're only storing 1 bit per cell instead of 4, your effective SLC capacity is 1/4 of the 500GB or 125GB. When the drive accepts an incoming write operation, the controller actually puts the data into this pseudo-SLC cache (where it can write quickly) and then during idle time it shuffles it over to the normal QLC. This avoids the pitfalls of QLC for almost all casual use. This is while the drive is *empty* though; as you fill up the drive, the 25% of unused capacity gets smaller and smaller, and can lead to somewhat catastrophic write performance falloff if that cache gets too small for day-to-day operations, which can happen if the drive is 99% full or something.

TLC drives do the same thing, their write penalty is simply smaller than QLC since it's 3 per cell instead of 4. MLC is 2 bits per cell and has a correspondingly lower write penalty, and true SLC has no write penalty, but essentially zero consumer SSDs are native SLC. TLC is kind of the performance 'sweet spot' where the penalty of writing direct to TLC isn't so bad, and the efficiency of offering a pseudo-SLC cache isn't as bad either at 1/3 capacity instead of 1/4.

The onboard DRAM cache on SSDs is mostly not used for cacheing incoming data writes, or I should say that isn't its most important job. it is instead used to cache an address mapping table. This table is somewhat analogous to a table of contents, it tells the flash drive where each of the bits are. DRAMless SSDs either have to make do without that table, which impacts performance, or some NVMe drives can use a protocol called HMB (host memory buffer) to borrow system memory to hold that table.
Great explanation.

Hmmm, how much? ;)

These drives will mostly be read only, which has great speeds, at least this one will be for games. Intel Optane seems to be the best drive for the system, will see what their pcie gen 4 ones go for once readily available.
 
So what I'm looking at there is no real noticeable real life performance difference. A classic case of 'nice benches, shame about the reality of it all'. Basically you can go as fast as you like but the differing file structures/sizes and file system will throw all that out the window. The issue aint hardware anymore. ;)

Law of diminishing returns. This is what I hate about PC tech now, it's all so fast and mature now. Was a different story back in 1992! I miss those days.

Back then you noticed an extra 33Mhz, 3FPS or 10MBps.
 
  • Like
Reactions: noko
like this
So what I'm looking at there is no real noticeable real life performance difference. A classic case of 'nice benches, shame about the reality of it all'. Basically you can go as fast as you like but the differing file structures/sizes and file system will throw all that out the window. The issue aint hardware anymore. ;)

Law of diminishing returns. This is what I hate about PC tech now, it's all so fast and mature now. Was a different story back in 1992! I miss those days.

Back then you noticed an extra 33Mhz, 3FPS or 10MBps.
Hey, I still notice the difference between 1Gbps and 100Mbps :). I really need to upgrade my local network to > 1gbe though as it's really becoming a bottleneck since 5 of my 6 desktops + my home server have SSDs. File transfers of 125MB/s are getting old when my slowest SSD can maintain 300MB/s (due to it being stuck on SATA 2, the drive itself can go faster).

It's awesome to see the QLC standing pretty strong against the TLC. I remember when people were saying to avoid TLC and stick with MLC because of endurance and speed reasons. Seems QLC is not to far off of TLC at this point and the $/GB is getting better. Not many people would notice on the day to day stuff between a 500MB/s SATA and a 5,000MB/s NVME.
 
the 25% of unused capacity gets smaller and smaller, and can lead to somewhat catastrophic write performance falloff if that cache gets too small for day-to-day operations, which can happen if the drive is 99% full or something.
Good breakdown!
But I often argue that writes to drives tend to get smaller and/or less frequent as a drive fills up to the point of near-intolerable transfer speeds, or at least they should in any user-controlled situation. If this is just a personal scratch drive, it should be treated like any other drive and kept under 80% capacity for headroom and performance.

Also, I've wondered something, and you might know the answer. Does over-provisioned space on an SSD that uses dynamic SLC caching use that provisioned space when calculating the total (in this case, the 25%)? I would assume the controller doesn't care where the free space comes from as long as it's good parts of the NAND. Also, don't SSDs still have factory-set provisioned space on the NAND? IOW, isn't their actual capacity higher than their addressable capacity?

I tend to not partition (over-provision) around 10-15% of my SSDs capacity anyway, though it's admittedly unnecessary, and mostly a psychological trick to keep myself from packing drives full.
 
Last edited:
Does over-provisioned space on an SSD that uses dynamic SLC caching use that provisioned space when calculating the total (in this case, the 25%)?
Most drive manufacturers don't spell this out, but I suspect the answer is "it depends". Over-provisioned space can be reserved for use as wear leveling spare area and not used as part of pseudo-SLC, or it can be used partially for pseudo-SLC, or a mix. Most consumer drives have very slim margins of reserved spare area, so I suspect it is not used for pseudo-SLC cache purposes, or perhaps not used for that unless the drive really needs it because it is 99% full or something.

Also, don't SSDs still have factory-set provisioned space on the NAND? IOW, isn't their actual capacity higher than their addressable capacity?
Absolutely. For consumer drives, this is normally very slim though - for example, most "1TB SSDs" have 1000GB of user-accessible space but 1024GB of NAND, with just the 24GB reserved as spare area. A huge exception to this was the Rocket Q 8TB drive that I recently reviewed, which has 8192GB of NAND but only 7680GB of user accessible space, leaving a relatively huge 512GB reserved area. In actuality, the 8TB drive should be marketed as a 7.68TB drive, which is how it would have been labeled as an enterprise drive. Enterprise drives are 480GB instead of 500, 960GB instead of 1TB, 1.92 instead of 2, etc; the amount of NAND is typically the same (sometimes even more) but a larger bit is carved away for reserved space.
 
Last edited:
Does over-provisioned space on an SSD that uses dynamic SLC caching use that provisioned space when calculating the total (in this case, the 25%)? I would assume the controller doesn't care where the free space comes from as long as it's good parts of the NAND. Also, don't SSDs still have factory-set provisioned space on the NAND? IOW, isn't their actual capacity higher than their addressable capacity?

By 25% he means full-drive SLC caching, the resultant SLC is just 25% the capacity of the drive as it's sourced from QLC. This can definitely come from OP space since you don't want to have 0GB of SLC when the drive is full, not least because you need some SLC at all times for things like the NAND copy of the FTL tables among other things (for example, some user data may be kept in there for reads). Most drives don't have full-drive SLC caching and instead rotate/shift the dynamic SLC from available native flash based on wear. This still happens with these drives - this is because dynamic SLC shares a wear zone with the native flash (GC etc). Likewise, less-worn SLC will be written first. This information is maintained in the block allocation table based on wear (# of erases, last time of access, and other related metadata). Static SLC, if present or also present, works differently as it's dedicated for the life of the device and is all in OP space. OP space has two components: physical, which is going binary to decimal, and marketed, which is any more than that. It's important to know that the actual amount of flash is not e.g. 256GiB (for 240/250/256GB drives) but rather much more than that as you have ECC/spare for example, RAID parity, have to compensate for bad blocks, etc. In any case, yes, SLC is logically addressed if it's dynamic and therefore shares the zone with the native flash which also logically shifts through OP space. Note that pSLC has an order of magnitude higher endurance than the native flash but dynamic SLC has an additive effect to wear via write amplification.
 
  • Like
Reactions: Nside
like this
It's awesome to see the QLC standing pretty strong against the TLC. I remember when people were saying to avoid TLC and stick with MLC because of endurance and speed reasons. Seems QLC is not to far off of TLC at this point and the $/GB is getting better. Not many people would notice on the day to day stuff between a 500MB/s SATA and a 5,000MB/s NVME.

It's definitely crucial not to fall into the trap of thinking flash levels are linear, e.g. SLC -> MLC -> TLC - > QLC with each one eventually "catching up" to the prior. Rather, the different types of flash are inherently manufactured differently - this is why QLC with just 33% more capacity per cell than TLC has dies that are two to four times as dense. It's also why QLC in pTLC mode - as with Toshiba's 96L QLC on the Inland Professional (new model) - is inherently less reliable than native TLC. SLC likewise these days tends to be oriented at SCM and ultra low latency (to compete with 3D Xpoint), and MLC is now disappearing from consumer drives (the just-announced 980 Pro is TLC-based). Of relevance here is simply that QLC has higher read, write, and erase latencies. A lot of your reads will be coming from the native flash, your writes will hit native flash outside SLC, and erase latency is an issue with garbage collection (as it write latency for that matter, including "folding" from SLC being slower). That is to say, these advances are more about "tricks" utilized in the controller and circuitry to get the most out of inferior flash, which can get quite complicated. It's actually rather neat.

You can see an article on that QLC here (the Toshiba stuff). Some examples of tricks include figures 13 & 14 illustrating how you can reduce the amount of programming steps for faster writes without introducing more errors. Figures 19 and 20 show why QLC reads so much slower and how they handle wordline transitions. Figures 28-29 show the independent plane reads (Intel also does this) plus the pTLC mode is shown in figures 25-27. You can compare this to Intel/Micron's 96L QLC, Samsung's 96L QLC, and Hynix's 96L QLC.
 
Last edited:
Those are some stupid speeds. What is thew cache size on those drives? once you fill that wont performance drop a lot?

Full-drive SLC caching on these (25/33% for QLC/TLC), which has significant drawbacks. It's also a bit unreliable in the current iteration. For the first point, it's because dynamic SLC has to be tracked and shifted around based on wear and further has terrible performance outside SLC if you manage to out-write it, but that's more of an issue when the drive is fuller (but if you're buying such a fast drive, aren't you getting it for writes?) For the second part, these drives utilize algorithms to manage SLC - this includes SLC behavioral profiles to match the workload (predictive) and also to reduce heat and power consumption. For some examples, Crucial's P5 will keep some user data (e.g. OS boot files) in the SLC cache for reads if it can plus it can change the size of the SLC dynamically - not as in, smaller as more space is used, but larger or smaller for a given amount used based on workloads. An example of power-saving would be going to direct-to-TLC/QLC mode rather than using SLC if you don't need that performance (quite common for consumer workloads). This is one of the reasons these drives tend to be inconsistent, though (ones with full-drive SLC caching).
 
Maxx knows his stuff! Though he may need reminding about what qualifies as "basics" since his basics document is 17k words long lol :)
 
This was very educational, as someone with 4 NVME drives in my computer, ranging from an older ADATA SC8200, to a QLC Silicon Power 1TB and also a newer Pci-e 4.0 Corsair MP600. It's a lot to take in.

I have 10% of every one of my NVME drives partitioned off as basically empty drive sections. I was hoping that would be enough to keep drive life longer and avoid overprovisioning.... Is that reasonable? Does it work like that?

Temperture goes over 85c when being stressed constantly with Crystal
I'm curious about this. Did you have any sort of heatsink on the NVME drive? I've never seen any of my drives get anywhere near 85c, I have, however, put those little $5 aluminum heatsinks on them. (Since moving to the X570 Taichi, I'm using the built in heatsink.)
 
Full-drive SLC caching on these (25/33% for QLC/TLC), which has significant drawbacks. It's also a bit unreliable in the current iteration. For the first point, it's because dynamic SLC has to be tracked and shifted around based on wear and further has terrible performance outside SLC if you manage to out-write it, but that's more of an issue when the drive is fuller (but if you're buying such a fast drive, aren't you getting it for writes?) For the second part, these drives utilize algorithms to manage SLC - this includes SLC behavioral profiles to match the workload (predictive) and also to reduce heat and power consumption. For some examples, Crucial's P5 will keep some user data (e.g. OS boot files) in the SLC cache for reads if it can plus it can change the size of the SLC dynamically - not as in, smaller as more space is used, but larger or smaller for a given amount used based on workloads. An example of power-saving would be going to direct-to-TLC/QLC mode rather than using SLC if you don't need that performance (quite common for consumer workloads). This is one of the reasons these drives tend to be inconsistent, though (ones with full-drive SLC caching).
Honestly, most of my drive use is heavily read oriented with the occasional writes. I would think most people are similar. Most of my writes are non-speed critical, meaning other things in my system are likely to be slower than the writes. For example, blender rendering. The render process takes much longer than it takes to write a frame to disk :). Or, compiling. It takes much more time to read all the files and produce the executable than it does to write it do disk. Games.. well, almost completely read. Windows, same deal, a few writes, but mostly reads. My server on the other hand is limited by my network speeds for most of my write work loads, although that may change eventually (better network, and some other work loads I have in mind for it that are a bit more write oriented). Still, great to see QLC doing so well. Each new generation needs time for better controllers and better processes to catch up, and a lot of things, like a gaming drive or similar, we don't need a huge write endurance or even awesome write speeds. Still, very valid points you bring up, it's important to know what you're using a drive for, but enudurance is normally not so much of an issue and as I said, most users have a lot more read's than writes.
 
  • Like
Reactions: noko
like this
I must admit I still do a little over-provisioning on my SSDs.

I just do

120GB - 1GB
240GB - 2GB
500GB - 4GB
1TB - 8GB and so on.

Mainly cos customers have a habit of filling stuff up and if it gives the controller a little breathing space all the better.
 
  • Like
Reactions: Nside
like this
Still, very valid points you bring up, it's important to know what you're using a drive for, but enudurance is normally not so much of an issue and as I said, most users have a lot more read's than writes.

Absolutely, it tends to be a 70/30 read/write mix, and those tend to be random. Although I think if you're getting a 4.0 SSD like these that you should be intending for such workloads and if not, something like the new P31 is superior in every way.
 
I have 10% of every one of my NVME drives partitioned off as basically empty drive sections. I was hoping that would be enough to keep drive life longer and avoid overprovisioning.... Is that reasonable? Does it work like that?

Generally consumer workloads aren't all that hard on a drive but there are exceptions, e.g. DRAM-less (particularly SATA/AHCI) or QLC-based. A good, DRAM-equipped TLC drive won't benefit much (AnandTech reviewed two E12 drives with different marketed OP - MP510 and P34A80 - and found virtually 0 difference in testing). I do have articles that cover the difference, e.g. performance and write amplification versus drive fill, but this is again dependent on the workload. In any case there's diminishing returns but a standard workload will see the vast majority of its benefit within just 15% of the drive being free. I don't want to overload you with my sources here but this is one example - the ratio is by which "total storage exceeds user-visible" or T/U, total blocks over user. 15% of typical 512GiB flash (480/500/512GB drive) would be ~445GiB, which means only about 5% of user space free on a 500GB SKU is necessary. This is an old article and is focused on WA but you get the idea - I usually suggest 20% of the expected total flash (that includes physical and marketed OP) is left free. Modern controllers can use free space dynamically as OP as GC/TRIM is quite aggressive by consumer standards (lots of idle time). Note that zone-based SSDs are a different discussion entirely...

20% would be, for example, 410GiB for 512GiB of flash, which if the drive is sold as 512GB (477GiB) would be 14% user free, for 500GB (466GiB) 12%, and for 480GB (447GiB) 8%. Possibly this is where the "10-15%" idea comes from some sources, although I stress it's dependent on many factors and there's no firm rule. Also, of course, dynamic OP is different than guaranteed, but again it depends on workload (e.g. idle time).
 
Last edited:
Those numbers look pretty standard for any PCIe4 drive based on the PS5016-E16 controller.

Here are the numbers from my Corsair MP600 1TB drive.

Drive temps are 49c to 63c Corsair drive comes with a good heatsink.

Corsair MP600 Crystal Mark 2.png


The newer PS5018-E18 is coming out which is even faster.

https://www.tweaktown.com/news/7484...lus-ssd-packs-insane-7gb-sec-reads/index.html

Those are some stupid speeds. What is thew cache size on those drives? once you fill that wont performance drop a lot?

On my 1TB drive the SLC cache is about 333GB's after that write speeds drop. I'm never usually writing that much data in sustained writes so I never hit it.
 
Last edited:
  • Like
Reactions: noko
like this
This was very educational, as someone with 4 NVME drives in my computer, ranging from an older ADATA SC8200, to a QLC Silicon Power 1TB and also a newer Pci-e 4.0 Corsair MP600. It's a lot to take in.

I have 10% of every one of my NVME drives partitioned off as basically empty drive sections. I was hoping that would be enough to keep drive life longer and avoid overprovisioning.... Is that reasonable? Does it work like that?


I'm curious about this. Did you have any sort of heatsink on the NVME drive? I've never seen any of my drives get anywhere near 85c, I have, however, put those little $5 aluminum heatsinks on them. (Since moving to the X570 Taichi, I'm using the built in heatsink.)
Looks like I am wrong with the 85c, redid tests and found at 80c the 1TB Sabrent Rocket Gen 4 drive apparently throttles over and over again using CrystalDiskmark with custom settings -> 16 Ques, 32 threads. To give you an idea why this particular SSD can get so hot to throttle point, I just have to show you. Yes the motherboard has a SSD cooler but not like what you have on the Corsair drive with fins, it is mostly just a plate with mass with some increase in surface area to dissipate the heat.

Here is image of motherboard plus some other stuff, the 1TB Sabrent Rocket Gen 4 m.2 drive is in the m.2 slot closest to the CPU, the one with the biggest cooler on it or plate:

BoardParts.jpg
The 1080Ti, or any two or more slot card will sit right on top of the m.2 slot and touch it:

M2DriveClearnaceGPU.jpg
Temperature tests comparing the motherboard m.2 mounted 1 TB Sabrent Gen 4 TLC SSD to the 2 TB Sabrent Gen 4 QLC SSD mounted on the MSI m.2 XPANDER-Z GEN 4 card:

Using Aida64 Stress test with disk option, after about 5 minutes the temperatures stablized, with the Sabrent Rocket Gen 4 1TB drive reaching 51c on the motherboard slot. The 2TB Sabrent Rocket Gen 4 on the card only 33c:

2020-08-31.png
The temperature can be seen to the left under temperatures for the two drives, C drive is the system drive, the 1TB TLC Sabrent listed as Sabrent Rocket 4.0 1TB in Aida 64. P drive is the 2TB Sabrent Rocket QLC drive listed as Sabrent Rocket Q4

The Sabrent Rocket 1TB TLC drive mounted on the motherboard reached 76c by just running the CrystalDiskmark benchmark:

2020-08-31 (4).png

Using custom settings in CrystalDiskMark I was able to get the drive to reach 80c and looks like it thermal throttles after this point, did not capture that part in this image, temperature would rapidly drop with decrease transfer rate:

2020-08-31 (5).png

The Sabrent Rocket 2TB QLC drive mounted on the Expander card reached 36c by running the CrystalDiskmark benchmark, cooling was very good with the cooling fan and much bigger heatsink, the temperature barely changed:

2020-08-31 (6).png

No matter what I did with CrystalDiskmark to stress the 2TB Sabrent Rocket Gen 4 QLC mounted in the MSI m.2 EXPANDER GEN4 card, the highest temperature was 42c:

2020-08-31 (8).png

At least in my setup, the motherboard m.2 slots could be limiting for Gen 4 drives that put out a lot of heat. The second slot has even a smaller cooler on it. I think there are motherboards that use the chipset fan to blow air on the m.2 slots or one of them. Meaning they are more than just chipset coolers. Still I am not sure I will run into this issue with normal use, but if games start to stream vast amounts of data to the GPU in the future, I think this issue could be a problem with Gen4 drives. Motherboard reviews I can see in the future should consider this aspect of a board design.
 
Last edited:
Great post lots of details.

I just checked my own temps with a crystal mark benchmark run.

And this is what I saw drive is in the M2 slot next to cpu.

corsair drive temp.PNG
 
  • Like
Reactions: noko
like this
around 61-63c is the throttle point on a lot of NVME SSDs (controller actual temperature is around 75-85c at that point) a lot of SSDs don't use the Controller internal temperature sensor to report on SMART they use the NAND temperature sensor witch can confuse people when there SSD drops in performance to the point it actually affects the system (my Toshiba XG3 does it when it hits 63c system is mostly unstable if good amount of reads or writes are going on)
 
  • Like
Reactions: noko
like this
around 61-63c is the throttle point on a lot of NVME SSDs (controller actual temperature is around 75-85c at that point) a lot of SSDs don't use the Controller internal temperature sensor to report on SMART they use the NAND temperature sensor witch can confuse people when there SSD drops in performance to the point it actually affects the system (my Toshiba XG3 does it when it hits 63c system is mostly unstable if good amount of reads or writes are going on)
On the Sabrent Rocket 1TB TLC Gen4 it is definitely 80c, it will reach that temperature (not sure where it is being measured) and hold it for around 5 seconds and you can see the read or write what ever it is doing dramatically drop with the temperature going down somewhat quick. Now are you saying there are different throttle points? That I maybe throttling even before 80c?
 
Sabrent sells a heatsink kit they recommend for their PCIe 4.0 drives, and it will knock 20+ degrees C off the temps if it also has a small breeze.
 
  • Like
Reactions: noko
like this
Sabrent sells a heatsink kit they recommend for their PCIe 4.0 drives, and it will knock 20+ degrees C off the temps if it also has a small breeze.
That would block using a card that would overlap the space above it like my graphics card. Which sits right on top of the motherboard m.2 heatsink/plate. The motherboard m.2 plate has a thermal pad which I removed the plastic protector when installing the m.2 but I need to check to see if it is actually contacting the SSD.

Stressing the GPU causes the m.2 to heat up as well but never saw anything over 50c from memory with the SSD inactive but with the GPU active. I could use that heatsink on the other m.2 slot if I was not going to use the 2nd pcie x16 slot wired for 16x, in reality I am going to put the second 1080 Ti there for now, so both motherboard m.2 slots will be covered by GPUs. The Xpander card works really well for cooling plus makes it easy to transport to another machine.
 
That would block using a card that would overlap the space above it like my graphics card. Which sits right on top of the motherboard m.2 heatsink/plate. The motherboard m.2 plate has a thermal pad which I removed the plastic protector when installing the m.2 but I need to check to see if it is actually contacting the SSD.

Stressing the GPU causes the m.2 to heat up as well but never saw anything over 50c from memory with the SSD inactive but with the GPU active. I could use that heatsink on the other m.2 slot if I was not going to use the 2nd pcie x16 slot wired for 16x, in reality I am going to put the second 1080 Ti there for now, so both motherboard m.2 slots will be covered by GPUs. The Xpander card works really well for cooling plus makes it easy to transport to another machine.
I was trying to look at your motherboards to see what you meant. Neither board in your signature supports PCIe4, so what's behind door number 3?

EDIT, Nevermind, I'm an idiot.
 
Last edited:
Back
Top