You didn’t veer off, that’s exactly what my question was about. 
And yes you raise a good point. Reminds me of what @Gio told me in another thread (see here) - he basically said that you don’t always have to follow the sheet music to the letter. (It was a different question, but kind of the same conclusion.)
I guess I need to build a bit more confidence before I start playing what I think is right instead of what the sheet music is telling me to play…