Scroll reader here. Had the same problem and after about a week my muscle memory has gotten used to it. However it IS a weird UX paradigm no matter how accustomed you get to it though so i feel your frustration still.
Incase you didn’t know, when you accidentally add the bookmark, if you’re in same general area of the book you can double tap the screen a second time to undo the bookmark.
It’ll grow on ya but i agree it’s still weird af