Performance issues - but only for first few minutes?

So I read through all the different threads, but I haven’t seen anyone mention this specific issue.

To start - my specs
GTX 980 - latest drivers
I5 3ghz
12 GB mem
Gigapower - so consistant 800 meg up and down.

My performance issues are a little weirder than everyone else’s. I’ve done the tricks. Turned down shadows, turned down the PhysX (even though they all default high), etc. My game runs with a framerate pretty poorly for about 5 minutes or so? Then all of a sudden? Smooth as glass. My guess is it’s trying to load and buffer things, and once it’s finished, it’s good to go. But with my high internet speed, I’m finding that hard to believe? So any ideas why it would be crappy one minute, and then great the next?

Just an update to hopefully help you guys out. It’s not EVERY game it’s happening in. So I just jumped into another match, and IMMEDIATELY it was smooth as glass. I normally know if it’s going to be bad as we’re waiting to be let out the gate, and everything chuggs along. But in this case, as soon as the match started I’m running around like a champ. Hopefully this helps you narrow in what is going on.