Like you said in the comment above,
You have 50 users (employees) on the same network as you and are accessing the same server as the one hosting your application.
Out of these 50 at least 10 are trying to access the same application.
So, here is what I think you should do,
During your work hours when all your staff is working run the same script with thread count of 50 users. While the test is running in JMeter, try to open the same link in a browser on your system. Does the website return any server error? If yes, the server is not able to handle the load and chokes.
Now when your test completes execution, again try to access the website in your browser. If it opens, then definitely the problem is server not able to handle the load.
Now about after work hours,
Once all the employees have left the office your test for 50 users works fine. Now increase the load gradually. Try 100 users. If it works fine try 150 or 200 users and so on. If the tests start to generate error. Adjust the load by decreasing the number of users and run again. Adjust the load till you find the point where the your tests start to fail. If the numbers match with that when you run your tests during work hours, then you know that the server chokes at a certain load.
If the numbers don't match, then you should be checking to see the Network I/O and see if your network chokes when there are a certain number of system generating load. This will tell you whether the outbound requests are failing on your network.