When your site is not getting heavy traffic does it suffer from any of these problems?
Could just be your hosting companies server setup.
Max allowed connections, slow hardware / bandwidth, without errors it is hard to know what the problem is. Knowing that it is only happening with high traffic is a good factor to look at your hosting company.