Nerve-racking vs nerve-wracking is a word I have always messed up. I tend to think that it’s spelled nerve-wracking but it’s actually spelled without the “w.” Think about how nerve-racking it is not to use a W.
The “mental torment” meaning of “rack” in “rack your brain” and “nerve-racking” comes from the idea of the physical torment of stretching bodies on the rack. Those are both spelled R-A-C-K.
Grammar Girl also notes the using “wrack” is archaic. “Wrack” is a sister of the word “wreck,” which is in relation to ships. So, unless you want to nerve-ship, use nerve-racking.