Test it both ways, especially if users are ever likely to encounter the situation where the DLL's have not been loaded up.
@Tristaan's answer goes into some good reasoning why you really ought to do it both ways.
That being said, in most performance or load tests we gradually ramp up the load because that's how it generally occurs in real life. So in most cases, the delays and issues with the initial loading of everything doesn't generally have a chance to affect the performance of a system in the real world as the load on the system increases.
About the only time a system will get suddenly slammed going instantly from nothing to full load is in a 'farm' situation which is running at high capacity and additional servers are spun up and brought online to help handle the load (something that can happen in a dynamically provisioned cloud environment). If that's something that could potentially happen in your case, then add that scenario to your tests as well.