What you are looking for is called static code analysis. (Check this wiki page for a list of tools.)
What type of testing tool can automatically test this?
One of the most famous tools is SonarQube, which supports Java, JavaScript, C#, and more. It also offers integrations for various build tools and CI servers (including Jenkins). Plus: It is open source.
And is it better to have it as a Jenkins hook or move it down and into the IDE the developer is using?
You probably want both. That is, some sort of code formatter within your IDE which is compliant with your tool's setting and the tool itself that checks everything that is pushed to the repository.
If, for instance, you use trunk-based development, you would break the build if you commit a style violation you didn't catch locally, since static code analysis tools typically mark builds unstable. And if you only check locally, you may integrate style violations when a developer has a misconfigured IDE. Therefore, you want to have both.