When it comes to the TDD cycle, how far through the testing should refactoring the code take place (incremental after each test or at the end)? And how would this affect the TDD cycle and the time depending on whether or not I refactor often?
First of all, since you mention "through the testing", I would like to remark that TDD is more a design methodology than a testing one. This means that you don't have a specific testing phase, because tests are defined as a first step and then all coding should be aimed to get those tests pass. "The testing" is a continuous process that will check every code contribution against existing tests to clarify if your code is improving or worsening the project goals achievement.
If you are working with sprints, your initial effort should be addressed to write new (failing) tests for your user stories and write the minimum necessary code to make them pass. Then, if you have spare time before the sprint ends and the next one comes, that would be the perfect moment for refactoring.
Anyway, this is not a universal truth and will change from one team to another. Just try to keep in mind what your main goal is and don't let refactoring tasks make you lose your track.
In addition, you can take a look on this question: https://stackoverflow.com/questions/255509/how-does-tdd-make-refactoring-easier