Softmod is for software modification. Changing your Cpu clock with a software that allows you to do it is an example.
Here for your Gfx card it means changing your Hardware ID so that your OS thinks it’s an other card. In your case you want your OS to think that it has a Quadro instead of a Geforce. It will allow you to install the Quadro drivers which are specificaly made for 3D apps.
After having a look at the specs of the Quadro Fx 3700 and 4700 I’d say that you should mod to the Fx 4700 as it is a Quadro with two G92 GPU. It seems to be the closest to your card. So I’d try 4700 first then 3700 if it doesn’t work. The only problem you may face if it works is that you only have 2 x 512 Gb VRam on the card which may be limiting for big scenes.
About your specs, you should be good with 16Gb Ram and 6 cores. When do you have viewport problem?
Last point just to be sure it’s clear for you so that you make the good choice is, like Gedd said, the importance of Vram (aka Video Memory). For the viewport, having too few Gfx memory can give a performance penalty if the scene is too big. Same goes when rendering with your Gfx Card. The entire scene must be contained in your Graphic Card Memory when in CPU only rendering mode. There may be a possibility of hybrid rendering with gpu and cpu but you’ll get a performance penalty and there is not always such option in every renderers. So having a lot of Vram is a good thing
As I said earlier, you may end up buying two cards depending on your scenes. Just be aware that the Quadro 2000 has only 1Gb Vram and that can also be limiting for OpenGL preview in Viewport if you go for it.
There is an interesting Review of different systems here
And for Softmodding you can get what you need here
Btw just asking, will you use Octane or Luxrender or an other software?