Whilst things did improve by fiddling with Windows Defender it wasn't a 100% fix which makes me wary. Interestingly, VS Code was always open when I did experience the issue and I haven't experienced it when it's been closed. So it may be the cause. I've opened an issue for this against the VS Code repo - it sounds like other people may be affected as I was. Perhaps this is VS Code and not Windows Defender. Watch that space...
The issue was VS Code. The bug has now been fixed and shipped last night with VS Code 1.14.0. Yay!