Do your requirements demand cross browser / device testing?
Some functional requirements do demand cross browser / device testing, e.g. an application needs to be logged in / out properly on all of its platforms.
Some functional requirements do not demand cross browser / device testing, e.g. a legal claim link that will pop out after you click it (a very lame example)
Which browsers are you thinking of?
There are so many browsers / devices out there, you can not test all of them. Your application may specify a preferred browser / device, you can focus your testing effort on it.
Do you have enough time to do it?
Ideally, the more browser / devices you test, the better test coverage will be. But it all comes down to its time box, do you have enough time to finish all of them? You can check with your stakeholders / business owner to verify time constraint.