I use Control+h/j/k/l for switching both tmux and vim splits, and haven’t touched the timeout. So far so good.
I did run into the delay when switching out of insert mode and immediately saving the file. Trying an auto command to save when exiting insert mode now, which helps with that.