Would I need to take down the original Coffee Shop Romances so Amazon doesn’t mark the new versions as plagiarism?

You'd have to take them down and when you publish the new versions, mention in the book descriptions that they're reworked versions of 'title of old book' (say the folks over at the indie author discord I'm part of)