Re-factoring code seems to be the more time consuming area of TDD, is there a way of knowing if you have spent too much time doing so? Or maybe a way of limiting the amount of re-factoring that must be done in order to continue testing?
Short answer: Limit refactoring to "removing duplication". This is Kent Beck's summary of the refactoring step in Test-Driven Development by Example.
Long answer: the time spent refactoring because of TDD is a blessing, not a curse. It's not a problem that TDD's refactoring takes time ... because in many languages refactoring is only possible because of TDD. TDD enables refactoring in an anxiety-free, predictable way. It's an amazing opportunity that doesn't come from other methodologies.
As you advance in your career, and work on problems you've seen before, you'll develop a feel for how much refactoring is necessary, and when.