One of the most frequently given pieces of engineering advice is something along the lines of "never fix something in the mix if you have the opportunity to get it right at the very beginning". Put another way, it's usually unwise to rely on EQ and other post-processing to get the sound you want. This is great advice, and wherever possible I like to mimic "real world" recording processes when I'm working digitally. To that end, paradoxically, I think it's often necessary to disregard, or at least re-interpret, that advice.
Why? Well, guitar amp modelling plugins improve by leaps and bounds year on year, but nothing's perfect, and I increasingly find that there are a few corrective processes I like to go through to make sure that the sounds I'm dealing with at mix time are the best they can possibly be. Essentially, this is a process of finding the quirks and weaknesses of my amp plugin and working to level them out until I have what is, to my ears, a good representation of the dry, pre-processed guitar sound should sound like.
In this article, I'll show you the processes I like to go through to achieve this, along with a few other tips that may help you get things sounding great.
Know Your Enemy
I've tried and owned more than a few of these products, and have begun to see the strengths and weaknesses of each. Magix Vandal, for example, strikes me as sounding very natural and dynamically responsive, but plagued by an odd mid-range quality. Guitar Rig, on the other hand, sounds fairly balanced but suffers from less impressive dynamics and a kind of pre-processed sheen which is hard to remove.
The point here is that, before seeking to correct any perceived problems with your modeller of choice, you need to know what you're looking for. To do that, download a few demo versions for unfamiliar modellers and A/B them with your regular squeeze. What are the good and bad points of each, and most importantly, what does this reveal about the characteristics of the one(s) you own?
Check The Basics
Before you go any further, be sure to check that you're giving your plugin the signal it needs. For example, I've found that different plugins respond better to different input levels: what works great for Guitar Rig might be too high for Amp Room, for example, so consult the product's manual and play around with your audio input settings to set an optimum level. If necessary, use a flat EQ or a gain plugin before the amp sim in your DAW's channel strip to set the input level.
Also, make sure that you're converting the impedance of your guitar signal. If you're using an input device with Hi-Z inputs, you're set, but if in doubt, route your signal through a disengaged, non-true-bypass effect pedal. A passive guitar's unconverted high impedance signal will sound flat and lifeless through your plugin.
Basics out of the way, we can get on with the job in hand. Set your plugin to produce the sound you want and, likewise, if you're in the habit of using 3rd party cabinet impulse files (e.g. Redwirez or ReCabinet) get that set up how you like it, too. Incidentally, if you're not using 3rd party impulses, they're well worth experimenting with: many, and especially the professionally recorded ones, can solve a lot of tone problems.
The next job is some post gain EQ. If your plugin will allow it, insert an EQ module between the pre and power amp modules; if not, put an EQ between the amp and cabinet models. Here, again, we're mimicking good "real world" practice. You can do a great deal to turn an OK guitar tone into a great one by throwing a good EQ in the effects loop of the amp.
Don't try to implement your mix EQ at this stage; no low cuts or dramatic sculpting of the mids. The idea is only to accentuate and refine the amp tone, rather than to radically re-shape it. If you feel like you want to make drastic boosts and cuts, it's probably better to rethink your amp, mic or cabinet choices – or all three.
Finally, it's time to find and deal with any unwanted artefacts of your modeller. Depending on how experienced you are at homing in on precise audio frequencies, the extent to which you're able to identify these problems will vary, but no matter how good your skills in that regard, the surefire way to home in on them is with a very narrow, extreme EQ.
You need a multi-band EQ which allows you to define the bandwidth, or "Q", of EQ adjustments. The basic channel EQ in your DAW will almost certainly do the trick. By setting a standard bell shaped EQ curve to a very high Q and an extreme amount of boost, you'll produce a very narrow EQ filter. Now, by slowly rolling this filter along the EQ bands, you should be able to home in on undesirable frequencies.
Very often, these undesirables will be higher frequencies, 5 KHz and beyond, but experiment and follow your ear. When you find offending frequencies, experiment with the amount of cut you apply: it's not always desirable to completely erase a given frequency, and going to far can leave your sound muffled and lifeless.
The main thing to remember here is to be sparing, and to maintain some perspective by keeping your ears fresh, so if you can, work at it little by little and keep refining it. Have fun!