Yes, there is a way possible to do this. You need to add a 'HTTP Request Default' configuration element from 'Configuration Elements' option i.e. Right click test plan and add this 'HTTP Request Default'.
In this 'HTTP Request Default' there is an option Connect in 'Timeouts (milliseconds)', you can specify your connection timeout value in this field and it will be applied to all child samplers. If this 'HTTP Request Default' is added at Test Plan level then it will be applied to all samplers and all thread groups.

If you want to specify individual Connection Timeout, then you can specify it in same field of each sampler. Individual sampler Connection timeout will override the 'HTTP Request Default' connection timeout value.

One more way is there but I haven't used that, you can specify this value in JMeter.properties file located at bin directory.
# set the socket timeout (or use the parameter http.socket.timeout)
# Value is in milliseconds
#httpclient.timeout=0
# 0 == no timeout
Uncomment the 'httpclient.timeout' property and specify its value. I have used the 'HTTP Request Default' solution and it works, but not used properties file solution, so you should check its effect using Table Listener before actually using it.
https://www.digitalocean.com/community/tutorials/how-to-use-jmeter-to-record-test-scenarios