You can go to 64GB but you must use special rams and remove all installed RAM
To have an idea go on crucial.com, select Apple and your computer and it will give you what you can put in your computer
example for my 12 core
You cannot mix Registered and Unbuffered memory. If you use the Crucial 16GB Kit (CT1421798) or 24GB Kit (CT1469655) all previous existing memory must be removed.
Memory Type: DDR3 PC3-10600, DDR3 (ECC)
Maximum Memory: 64GB
Slots: 8
Each memory slot can hold DDR3 PC3-10600 with a maximum of 12288MB per slot.*
*Not to exceed manufacturer supported memory.