AMD cc:NUMA support...

[ex_forum_user_3]

Hi guy's,

If you have a board that supports ccNUMA, turn it on!!!
Boards that support this are:

K8D Master (series) with 2 CPU's
K8D Master3 (series) with 2 CPU's

You need to have min 2 sticks of memory installed, but better is 4, I talk further about 4 sticks.
2 sticks connected to each CPU.

Now you need to go into the BIOS and turn Node (Memory) Interleave OFF!
Do not confuse this with Memory Interleave, that's per CPU and should stay on.

You need to turn this off, because it should not automaticly use any memory that it can find.

Further you need WinXP SP2! (or Win2000 Datacenter or simular)

Check your memory speed, normaly Sandra or something should show a normal speed for your memory.
E.g.:

Normal Dual channel -> PC-2700x2 is about 4800MB/s (just a sample!)
Even with 2 CPU's and 4 sticks installed (2 for echt CPU)
Sandra shows NUMA not supported.

Now edit your boot.ini and add the /PAE switch!
This is very important, as that will turn your Windows to use ccNUMA.

Then it means that every single program will be using it's closest memory.
And share if needed the speed from the other CPU's memory.
Good written programs, multi-threading can use this to improve a lot.

Now check with Sandra memory bench again!

It should go upto 9000MB/s or such.
Also Sandra should show NUMA as being supported.

This should speedup your system under Windows a lot.
Linux can also use this, but I don't know how to check it.

Anyway, it will give your system a whole lot more memory bandwidth to play with.
This only works on the boards listed, as they have memory connected to each CPU.

The K8T-Master or K8N-Master series with 2 CPU's can't do it because they have only memory connected to CPU1.

Windows Vista should have this by default, but I'm not sure, you need to Google on that.

 

[ex_forum_user_3]

Well a little update, the K8N Master can do NUMA too according to Sandra.

The /PAE switch should be enough.

For other boards that have memory per CPU.

Disable Memory node interleave.
Turn SRAT on in the BIOS.
Edit the BOOT.INI and add the /PAE switch.

Now reboot and NUMA should be working, but you need at least WinXP SP2 or higher.
Beware, if you don't disable Memory Node Interleave, NUMA will not work!!

NUMA does make a big difference, so use it if you can.

The following AMD boards have NUMA full blown:

K8D Master-133 series
K8N Speedster series
And other DUAL AMD CPU systems that have memory connected to each CPU.

The K8N Master seems to support it, but I'm not quite sure what it does with it, maybe somebody can test and find out.

 

[harshvfx]

I understand everything except how to add the PAE switch into the boot.ini, can you give me quick detail on how to?

 

[Svet]

I understand everything except how to add the PAE switch into the boot.ini, can you give me quick detail on how to?

look here:
https://forum-en.msi.com/index.php?topic=104877.msg790762#msg790762

 

[Hans]

K9ND Master/Speedster does support ccNUMA as well.

 

[Pluk]

Hi Bas,

you stated that the K8n Master2-FAR seems to be able to do NUMA according to Sandra, but I can't seem to find the BIOS settings required to do it. I've set Node Memory Interleave to disabled, but where is the SRAT setting ? Without that setting Linux (OpenSuse 11.0 2.6 kernel) doesn't do numa. numactl --harware shows only 1 node is available. numademo shows about 2700MB/s of memory throughput, as I've only got PC2100. All 6 slots are filled though, so I'd like to increase the memory performance if possible.

I am running BIOS 1.42t (yes probably am the only one in the world running that particular version ;-)

Regards,
Pluk

 

[ex_forum_user_3]

It can do it but it won't do much good, as only CPU0 is connected to memory and CPU1 must retrieve it from CPU0.
It does work, but you gain nothing
There are only gains when you have a board with memory connected to each CPU.