PCIE Bifurcation

x0 means 0 PCIe lanes connected to the second slot. This is because the splitter will route lanes 0-7 to the first slot, and 8-15 to the second slot. But in a mechanical x16 / electrical x8 slot, only lanes 0-7 are connected.

To achieve what the previous poster wants, an x8 to dual x4 splitter is necessary.
 
2)no, one will work at x8 and the other one will get x0
So, if my goal is to get 8 PCIE cards working off a single motherboard, instead of trying to use every available slot and then splitting the last one one or two, I should instead only use slot combination that motherboard can provide with 16 lanes per slot and then use two of your x4x4x4x4 splitters? I don't see any x8 to dual x4 splitters with power connectors. This approach actually widens the motherboard selection quite a bit.

Or, make sure the splitter is in the slot that stays 16x when all slots are used.
 
I do have an x4x4 card. https://peine-braun.net/shop/index.php?route=product/category&path=65_67
An x4x4x4x4 in the first slot would still be preferable I guess.
Your BIOS/CPU must be capable of supporting this though. Need to verify that first!
Damn, how did I miss your store link. I see you even have PLX splitters there, I wonder how those would show up in VFIO grouping for passthrough.

Those PLX splitters, let's say I am crazy and want to connect multiple PCIE cards to Intel Nuc that has an M.2 slot internally. I probably can buy an M.2 to PCIE x4 adapter (i.e. EXPLOMOS M.2 Key M NGFF to PCI-E 4X Adapter Card) + your PLX splitter and do it? Would it work with x4 PCIE? What about a similar stunt but with external thunderbold eGPU that also provides 4 lanes according to GPU-Z.
 
Last edited:
How difficult would it be to make something like this but in M.2 format (x4 -> x4x4 or x4->x4x4x4x4 for instance)?

Physical placement of all necessary components and the temperature of the PLX chip.
Need a smaller device, with less power consumption. (less power rails as well!)
Would the temperature and size be the only problem here? I am seriously tempted to try connecting your bigger PLX board to an M.2 slot via an ADT M.2->PCI x4 cable unless you can tell it is not going to work on just 4 lanes right away.
 
C_Payne, I received the x8/m.2/m.2 riser and put it through its paces. It works beautifully!

20210218_214413.jpg

20210218_214429.jpg


Full passthrough for pcie x8 and a pair of m.2s in a low profile footprint. It's a work of art.
 
C_Payne, I received the x8/m.2/m.2 riser and put it through its paces. It works beautifully!

View attachment 330910
View attachment 330911

Full passthrough for pcie x8 and a pair of m.2s in a low profile footprint. It's a work of art.
Curious how this would ideally be installed? Love the product of course, super neat stuff. This is the one designed for the NCASE I believe. Any pics of the install by chance?
 
Curious how this would ideally be installed? Love the product of course, super neat stuff. This is the one designed for the NCASE I believe. Any pics of the install by chance?

It's actually designed for the U-NAS NSC-800, but should work in any other situation where you want to run a low-profile x8 card and still use the rest of the lanes. It's a tiny NAS case that fits a mini-itx motherboard and 8x 3.5" drives.



The bifurcating riser is connected to the motherboard, and a short length of flex riser goes from that riser to a LSI9211-8i SAS2 card installed in an offset position. Across the top is a Mellanox Connectx-3 in a 3d-printed retainer bracket, connected to the motherboard's m.2 slot through a m.2 to pcie x4 adapter.

In total, in that small case, I have the motherboard/cpu/memory, 8x 4TB SAS2 drives, 2x 2.5" SSDs, 2x nvme drives, a SAS2 card, a Connectx-3 card and a pair of 120mm fans.

About as packed as it can get for a case that's 12"x10"x7" (~13.7 liters)
 
It's actually designed for the U-NAS NSC-800, but should work in any other situation where you want to run a low-profile x8 card and still use the rest of the lanes. It's a tiny NAS case that fits a mini-itx motherboard and 8x 3.5" drives.



The bifurcating riser is connected to the motherboard, and a short length of flex riser goes from that riser to a LSI9211-8i SAS2 card installed in an offset position. Across the top is a Mellanox Connectx-3 in a 3d-printed retainer bracket, connected to the motherboard's m.2 slot through a m.2 to pcie x4 adapter.

In total, in that small case, I have the motherboard/cpu/memory, 8x 4TB SAS2 drives, 2x 2.5" SSDs, 2x nvme drives, a SAS2 card, a Connectx-3 card and a pair of 120mm fans.

About as packed as it can get for a case that's 12"x10"x7" (~13.7 liters)

That's incredible. I've been wanting to do something similar and yours is certainly a great example to strive for. I was actually just made aware of that NAS case. Shame its not sold anymore by the looks of it. Might have to bite on some of the ebay listings I see of it even with their markup. For 8x 3.5 HDDs and as much functionality as you've been able to squeeze out of such a small case, it seems like a no brainer. Thanks for sharing!
 
Shame its not sold anymore by the looks of it.

AFAIK it might just be out of stock? Could be worth contacting them to see if they are going to restock soon. The NSC-810 and 810A were released after I got it and might be a bit more convenient. The 810 takes a mini-ITX board and doesn't need a flex riser, while the 810A can handle micro ATX boards.
 
I had never heard of the U-NAS cases before, but they are pretty sweet. I like the 810A idea, and also maeslin, your build out is awesome!
 
I had never heard of the U-NAS cases before, but they are pretty sweet. I like the 810A idea, and also maeslin, your build out is awesome!

Thanks! Here's pictures of the full build for reference:


One little word of advise with those cases; they tend to have some pretty sharp edges inside. Get some grommet edging ('grommet strip') if you get one of them, it'll save both cables and fingers.
You can see what I mean surrounding the holes where the power cables and SAS cables feed through in pictures 1, 15, 33, 35 and 37.
 
C_Payne Can I get your recommendation on what riser to use with your x8x4x4 in the nCase M1 please?

You mention "R83SR" on your site, but I can't find one on their page: http://www.adt.link/x16.html

There seems to be two possibilities on http://www.adt.link/product/R83.html

RS83R - I think a 10cm one could bend straight down from the second slot on the riser, go underneath the GPU, and bend over in a way that allows getting the horiztonal placement of the slot right
RS83L - A short one (maybe 5cm) might work too - but I fear it means the loop now has to go up in between the main GPU and low-profile card.
 
C_Payne Can I get your recommendation on what riser to use with your x8x4x4 in the nCase M1 please?

You mention "R83SR" on your site, but I can't find one on their page: http://www.adt.link/x16.html

There seems to be two possibilities on http://www.adt.link/product/R83.html

RS83R - I think a 10cm one could bend straight down from the second slot on the riser, go underneath the GPU, and bend over in a way that allows getting the horiztonal placement of the slot right
RS83L - A short one (maybe 5cm) might work too - but I fear it means the loop now has to go up in between the main GPU and low-profile card.
Uhm, I just checked their page again. It's "R83SR" I can't find the "RS83R". I think they are the same thing though, maybe a misspelling.
Make sure to use the "R" one and not the "L" one as that one won't fit.
 
Uhm, I just checked their page again. It's "R83SR" I can't find the "RS83R". I think they are the same thing though, maybe a misspelling.
Make sure to use the "R" one and not the "L" one as that one won't fit.
You're right, sorry not sure why I put the 'S' in the wrong place.

So you're saying the R83SR, presumably because it can be folded over to get the right spacing, while then putting the slot in the right orientation?
 
You're right, sorry not sure why I put the 'S' in the wrong place.

So you're saying the R83SR, presumably because it can be folded over to get the right spacing, while then putting the slot in the right orientation?
Yes, the left turn one will interfere with the mobo.
 
Hey C_Payne! Are you planning to make the board you made for maeslin a regular in-stock item in your shop? I'd like to buy it but I can't justify the price for a custom order.

Does it support PCIe Gen 4 speeds for all three slots? Would you need a Gen 4 redriver?

Thanks!
 
I will have them back in stock in 3 weeks or so. gen4 should be good, if you are not planning to add a cable.
 
I will have them back in stock in 3 weeks or so. gen4 should be good, if you are not planning to add a cable.
That's awesome!

I plan to use a GPU with a Gen 4 riser cable on the x8 slot, so it sounds like I'd need a redriver for that. In this case, should the redriver go into the x8 slot or directly into the motherboard?

Thank you
 
you may be lucky with a very high quality and short cable, like the loque cobalt, when you plug it directly into the riser and the GPU on an ITX mobo.
 
C_Payne, I received the x8/m.2/m.2 riser and put it through its paces. It works beautifully!

View attachment 330910
View attachment 330911

Full passthrough for pcie x8 and a pair of m.2s in a low profile footprint. It's a work of art.
That adapter is amazing!
What if it were used with two of these:
https://www.amazon.com/gp/product/B08CX7FWHC/ref=ppx_yo_dt_b_search_asin_title?ie=UTF8&psc=1

61yaEXMXbaL._AC_SL1001_.jpg

I'm using one of these right now on the thin client in my sig with a standard PCI-E to M.2 (PCI-E 4x) adapter, but with the adapter above, that could potentially allow for up to three GPUs or PCI-E devices off of one 16x slot, assuming one of these on each side, plus the slot on top. (y)
 
As an Amazon Associate, HardForum may earn from qualifying purchases.
That adapter is amazing!
What if it were used with two of these:
https://www.amazon.com/gp/product/B08CX7FWHC/ref=ppx_yo_dt_b_search_asin_title?ie=UTF8&psc=1

View attachment 340401

I'm using one of these right now on the thin client in my sig with a standard PCI-E to M.2 (PCI-E 4x) adapter, but with the adapter above, that could potentially allow for up to three GPUs or PCI-E devices off of one 16x slot, assuming one of these on each side, plus the slot on top. (y)
Running multiple GPUs off a single M.2 slot (4 lanes) is a bad idea.

Bifurcation is out of the question: no 4x->??? risers that I know of, and I highly doubt your motherboard has support for it.

That leaves PLX but it would be very slow AND very expensive.
 
As an Amazon Associate, HardForum may earn from qualifying purchases.
Running multiple GPUs off a single M.2 slot (4 lanes) is a bad idea.

Bifurcation is out of the question: no 4x->??? risers that I know of, and I highly doubt your motherboard has support for it.

That leaves PLX but it would be very slow AND very expensive.
It wouldn't be doing that, though, GPU0 would be on the PCI-E adapter itself (4x), then GPU1 and GPU2 would be on the two side M.2 slots to that adapter I posted (each 4x).
The power draw would primarily being going through the additional PSUs for the adapters.

Also, I'm only using the one adapter that I posted, not the whole thing - that was just a suggestion and possibility, and I see no reason it wouldn't work if a motherboard had support for bifurcation.
I never said my motherboard specifically had support for it, which it doesn't since it is only PCI-E 2.0 4x.
 
It wouldn't be doing that, though, GPU0 would be on the PCI-E adapter itself (4x), then GPU1 and GPU2 would be on the two side M.2 slots to that adapter I posted (each 4x).
The power draw would primarily being going through the additional PSUs for the adapters.

Also, I'm only using the one adapter that I posted, not the whole thing - that was just a suggestion and possibility, and I see no reason it wouldn't work if a motherboard had support for bifurcation.
I never said my motherboard specifically had support for it, which it doesn't since it is only PCI-E 2.0 4x.
I don't think a 16x->? bifurcation riser (like the one maeslin has) would work on a 4x slot (like the one on a M.2 -> PCIE 4x adapter). 12 of those 16 pins are disconnected.
 
I don't think a 16x->? bifurcation riser (like the one maeslin has) would work on a 4x slot (like the one on a M.2 -> PCIE 4x adapter). 12 of those 16 pins are disconnected.
I think I am transposing my words, and you are right.
The primary adapter would have to be on a 16x (mechanical and electrical) slot for the 4 lanes on top and the 4 lanes for each side (8 total) for m.2 to work.

It would not work on a 16x mechanical and 4x electrical slot, for sure.
 
Also it would be simpler to use my x8x4x4 and a single x16 cable I assume. less interconnects, less singal issues.
 
Hi!

Did any of you experiment with PCIE -> M2 and M2 -> PCIE? I am curious if it can be done. Many cheap A320 and H310 ITX boards have only 1x16 and 1xM2, but the M2 can run in PCIE x4 mode, so in theory it is possible to get another x4 slot from it. ADT-Link has cables for this, but I'd like to know how reliable they are. Do they always work if the motherboard has a "M2 PCIE x4" mode?
Another thing that there are cheap B450 boards with 1x16 -> 2x8 birfucation, at least there is one: ASRock Fatal1ty B450 Gaming-ITX/ac. So in theory with this board I could have 2x8+1x4 with PCIE bifurcation and M2->PCIE conversion. I am curious how the bifurcation part can be done and how expensive it is? This is only a $125 board, so it would be weird to spend $125+ on 2x8 bifurcation instead of buying a $250 board.

(I am about to build a router between my PC and home server and it would be nice to have something compact like mITX. It should have two QSFP+ ports and one SFP+ or 10Gbps RJ45 port, so this would take PCIE 2x8+1x4 total. I would rather spend more on the network cards and the CPU than on the motherboard. But if there is no cheap solution, then I guess I have to give up compactness and go with a PCIE 2x16 uATX and the same M2 -> PCIE conversion, which hopefully works.)
 
Last edited:
C_Payne - I'm trying your bifurcator for the first time in a new way - passing through two different GPUs, to two different VMs, at the same time (whereas previously the guest had one, and the host had one).

What's really peculiar is that I can passthrough both GPUs to a *one* VM fine. But if I passthrough to two different guest VMs, I get an instant crash of the machine.

I've ensured BIOS set to Gen 3 PCI. IOMMU groups show the GPUs in different groups.

If it were a power issues presumably that wouldn't be anything to do if it were one VM or two. So I'm really scratching my head. There's nothing to go on in the logs - and just completely and instantly kills the host OS.

One thing I did try was enabling 'AER' in the BIOS, although I'm not exactly sure what that does. When I turned that on my guest logs are spammed with correction logs:

Code:
[ 2268.015030] pcieport 0000:00:03.1: AER: Multiple Corrected error received: 0000:07:00.0
[ 2268.015037] pcieport 0000:00:03.1: AER: PCIe Bus Error: severity=Corrected, type=Data Link Layer, (Transmitter ID)
[ 2268.015041] pcieport 0000:00:03.1: AER:   device [1022:1483] error status/mask=00001000/00004000
[ 2268.015045] pcieport 0000:00:03.1: AER:    [12] Timeout
[ 2268.015051] vfio-pci 0000:07:00.1: AER: PCIe Bus Error: severity=Corrected, type=Physical Layer, (Receiver ID)
[ 2268.015055] vfio-pci 0000:07:00.1: AER:   device [10de:1aef] error status/mask=00000001/00000000

IOMMU groups:

Code:
IOMMU Group 18:
    07:00.0 VGA compatible controller [0300]: NVIDIA Corporation Device [10de:2206] (rev a1)
    07:00.1 Audio device [0403]: NVIDIA Corporation Device [10de:1aef] (rev a1)
IOMMU Group 19:
    08:00.0 VGA compatible controller [0300]: NVIDIA Corporation GP108 [10de:1d01] (rev a1)
    08:00.1 Audio device [0403]: NVIDIA Corporation GP108 High Definition Audio Controller [10de:0fb8] (rev a1)
Attach files
 
C_Payne - I'm trying your bifurcator for the first time in a new way - passing through two different GPUs, to two different VMs, at the same time (whereas previously the guest had one, and the host had one).

What's really peculiar is that I can passthrough both GPUs to a *one* VM fine. But if I passthrough to two different guest VMs, I get an instant crash of the machine.

I've ensured BIOS set to Gen 3 PCI. IOMMU groups show the GPUs in different groups.

If it were a power issues presumably that wouldn't be anything to do if it were one VM or two. So I'm really scratching my head. There's nothing to go on in the logs - and just completely and instantly kills the host OS.

One thing I did try was enabling 'AER' in the BIOS, although I'm not exactly sure what that does. When I turned that on my guest logs are spammed with correction logs:

Code:
[ 2268.015030] pcieport 0000:00:03.1: AER: Multiple Corrected error received: 0000:07:00.0
[ 2268.015037] pcieport 0000:00:03.1: AER: PCIe Bus Error: severity=Corrected, type=Data Link Layer, (Transmitter ID)
[ 2268.015041] pcieport 0000:00:03.1: AER:   device [1022:1483] error status/mask=00001000/00004000
[ 2268.015045] pcieport 0000:00:03.1: AER:    [12] Timeout
[ 2268.015051] vfio-pci 0000:07:00.1: AER: PCIe Bus Error: severity=Corrected, type=Physical Layer, (Receiver ID)
[ 2268.015055] vfio-pci 0000:07:00.1: AER:   device [10de:1aef] error status/mask=00000001/00000000

IOMMU groups:

Code:
IOMMU Group 18:
    07:00.0 VGA compatible controller [0300]: NVIDIA Corporation Device [10de:2206] (rev a1)
    07:00.1 Audio device [0403]: NVIDIA Corporation Device [10de:1aef] (rev a1)
IOMMU Group 19:
    08:00.0 VGA compatible controller [0300]: NVIDIA Corporation GP108 [10de:1d01] (rev a1)
    08:00.1 Audio device [0403]: NVIDIA Corporation GP108 High Definition Audio Controller [10de:0fb8] (rev a1)
Attach files
AER = advanced error reporting
 
I read a lot about bifurcation, but the basics are still not clear for me. Can somebody answer these?

1. Do the SATA and M2 SSD-s communicate with the same lanes with the CPU or does the motherboard have sort of extra lanes for them? If so, then how do I know how many extra lanes they use?
2. If the motherboard has M2 slots, then can I convert those into full speed x4 slots e.g. for a network card or will they run only low speed?
3. If the motherboard has two x16 slots and supports for example x16 for the first and x8 for the second, then can I do 2x8 bifurcation for the first slot and use 3 cards with x8 connectors? Or should the motherboard manual explicitly write if this is possible?
4. If an mITX board supports x16 -> 2x8 bifurcation, then do I need an extra bifurcation riser card on that board for $100 e.g. QIP4X-PCIE16XB01 to make it really work or is it just about cabling and it does not need extra electronics? Do these noname cards even work? Somehow I cannot find a decent manufacturer which sells PCIE risers, I mean something like Asus or ASRock, which sells motherboards too...
5. Is using PCIE x16 -> 4x M2 -> PCIE 4x4 a legit way to split up an x16 port or better to use x16 -> 4x4 bifurcation risers? Is it possible to split up and x16 slot with a PCIE -> M2 adapter even if the motherboard does not support bifurcation for that slot?
 
Thanks inf3rno - what I'm not clear on is if AER is uncovering, or causing issues? I mean, I have the problem before enabling AER, so wondering if this is explaining it?
If it is what the name says then it does not cause issues, it just reports more errors if you turn it on. I guess the corrected errors are not a big issue and you need to look for the not correctable errors, but I don't know much about what's normal in the case of PCIE.
 
If it is what the name says then it does not cause issues, it just reports more errors if you turn it on. I guess the corrected errors are not a big issue and you need to look for the not correctable errors, but I don't know much about what's normal in the case of PCIE.

I just tried this - I removed the bifurcator, turned the BIOS back to x16, and the AER error disappear (even though AER is still enabled).

When I try x8x8 and just have one GPU (no nvme drive or second GPU), the AER errors return. And not just a few - the logs are completely full of them.
 
I just tried this - I removed the bifurcator, turned the BIOS back to x16, and the AER error disappear (even though AER is still enabled).

When I try x8x8 and just have one GPU (no nvme drive or second GPU), the AER errors return. And not just a few - the logs are completely full of them.
What kind of card do you use? Do you have this error only in VM?
 
I’m using AmiBcp 5.02 and on my board there really is no security. I’m just confused why there are four IO divisions with each one having their own IOU 0-2 controller reference and each one of those four IO divisions have 11 pcie slices. If you find 5.02 take a look.

Thanks
Wait, does this mean it is possible to add bifurcation to mobos with Intel chipsets too? Or is it just that in the case of some boards the manufacturer was too lazy to enable it, but it is already supported for boards with the same chipset?
 
I just tried this - I removed the bifurcator, turned the BIOS back to x16, and the AER error disappear (even though AER is still enabled).

When I try x8x8 and just have one GPU (no nvme drive or second GPU), the AER errors return. And not just a few - the logs are completely full of them.
I just asked because if you don't have the error in the logs outside the VM, then it can be a VM related issue, but if you got them, then I think this is hardware related and your card is junk. At least 90% chance, that we are talking about the latter case. Actually this is what I am struggling with too, I cannot find a decent 2x8 passive card in the EU, all I could do is ordering one from C_Payne here and who knows how bad shipping prices, customs, VAT, etc. would be.
 
Hello, Tilde, I am already give up to use ameri-rack to retrieve a flexible 2*8 risers, I am also have the requirements to getting 4*10Gb SFP+ and also 4*1Gb POE(48v) ports.
And Now I am not using flexible riser but using a fixed 2*8 riser.
I'd like to post a URL for you to buy, but comes from China.
And besides, I am also ask for someone designing a custom ITX-box for two 2*8 PCI-E card(Full height).
You may be interesting:)
View attachment 149052
View attachment 149053 View attachment 149054
View attachment 149055
This Chinese stuff looks like exactly what I need: https://www.ruten.com.tw/item/show?22030929704581 But $1200 is a joke for a passive card. Are they nuts? Or is it $1.2 lol? Quality does not look like the best though.
 
Ok I read the entire topic in the last 3 days. I can answer some of my questions:

1. Do the SATA and M2 SSD-s communicate with the same lanes with the CPU or does the motherboard have sort of extra lanes for them? If so, then how do I know how many extra lanes they use?
-> there are extra lanes for them, it depends on the chipset how many, usually x4 I think, but better to check it in the manual of the mobo chipset
2. If the motherboard has M2 slots, then can I convert those into full speed x4 slots e.g. for a network card or will they run only low speed?
-> it is possible of the mobo supports PCIE x4 mode for the M2
3. If the motherboard has two x16 slots and supports for example x16 for the first and x8 for the second, then can I do 2x8 bifurcation for the first slot and use 3 cards with x8 connectors? Or should the motherboard manual explicitly write if this is possible?
-> I think the mobo manual should explicitly mention this, maybe with a modified BIOS this is possible, but one needs to check the available BIOS with modding software, or ask the manufacturer to modify the BIOS for them. note that is is possible only if the chipset allows it.
4. If an mITX board supports x16 -> 2x8 bifurcation, then do I need an extra bifurcation riser card on that board for $100 e.g. QIP4X-PCIE16XB01 to make it really work or is it just about cabling and it does not need extra electronics? Do these noname cards even work? Somehow I cannot find a decent manufacturer which sells PCIE risers, I mean something like Asus or ASRock, which sells motherboards too...
-> one needs a passive riser without PLX chip for this, probably the card that I linked works. ASRock manufactures bifurcation riser cards, but only for servers probably only for companies and I guess it is very expensive. I could not find any of them in ebay. https://www.asrockind.com/en-gb/PCIE-Riser-2S risers appear to be very rare nowadays. I could not find supermicro or amerirack cards on ebay or amazon I could use for 2x8, maybe I could ask amerirack to send me one, but I would rather use a proper webshop, than sending mails. the best bet is buying a custom built riser something like C_Payne manufactures. https://peine-braun.net/shop/index.php?route=product/category&path=65_59 or there is another guy who is doing it too: https://riser.maxcloudon.com/de/bifurcated-risers/25-bifurcated-riser-x16-to-2x8-set.html I whined a lot to ASRock in a mail about why they don't make consumer bifurcation risers when they support bifurcation in consumer mobos, I doubt thought it will have any effect. it is a pain to find bifurcation riser cards especially cheap ones.
5. Is using PCIE x16 -> 4x M2 -> PCIE 4x4 a legit way to split up an x16 port or better to use x16 -> 4x4 bifurcation risers? Is it possible to split up and x16 slot with a PCIE -> M2 adapter even if the motherboard does not support bifurcation for that slot?
-> I think this is possible, though it might be more expensive than a PCIE only solution. it has the advantage that there are already such M2 risers e.g. Asus makes one. it is relative cheap, but the board needs to support 4x4 on that slot. even some Asus boards support 4x4 because of this. only with a PLX chip would be possible to do this without motherboard support. As of the boards, ASRock or Gigabyte is the best bet for me, where the manual says 2x8 riser card support. MSI might work with a modded BIOS, but I don't want to tamper with BIOS and potentially brick the board. Asus will never support 2x8.

I still have no idea which 2x8 passive riser to buy, it is really hard to get them here in Europe with a decent price and I am cheap atm. I might buy an uATX board with two x16 slots and x8 mode support for the second slot and do the M2 -> x4 conversion. It is a lot cheaper and it does not need this kind of magic, but it is hard to give up compactness...
 
Back
Top