I'm looking for the term used to describe when the amount of cumulated hacks and workarounds have made a software code too expensive to maintain at a reasonable level of quality.
Consider Ward Cunningham's technical debt metaphor. It isn't exactly what you describe. Technical debt is more the accumulation of hacks and workarounds, even if the accumulated expense is not yet "too expensive to maintain."
But it seems like it's in the ballpark of what you're asking about.
Robert C. Martin also known as UncleBob has coined a term for this "Clean Code" the other way around is what you are describing and unmaintainable, un-extendable messy piece of code that smells. This would be unclean-code or messy-code, but also known as QuickAndDirtyCodeTM.
UncleBob wrote a book (Clean Code) about it that is a must read for any developer if you ask me. Be sure to watch his video explaining what types of unclean code you have and how it can ruin companies. He is a bit excentric, but if you can view past that it is a great watch.
This leads to "code smells", these are recongizable piece of code that match one or more known code smells. Martin Fowler wrote a book (Refactoring) about how to change smelly code in to better clean code.