Good news: LO 18.104.22.168 now respects the RLO on Windows 7. I was using my Old Italic font to enter text, but then LO switched to a different font after I inserted the RLO (RLO is included in my font). Reapplying my font caused the characters to be flipped horizontally, so <rtlm> is automatically applied if the font has it. I'll send you a PM with details about the font change issue since they are probably off topic for this thread.
The bug you mentioned is interesting for those of us who work with historic scripts -- an historic RTL script used for a language that now is written LRT. No wonder the software got confused. I don't know of any parallel case.
This thread is a bit old but I caught up with the changes in LibreOffice 5.3 only a couple of weeks ago. I am very pleased with, and very grateful for, this development, even if it is a byproduct of other changes (as Khaled said).
In fields like classics and medieval studies, the issue of glyph variants frequently comes up. For a very long time I have wanted to encourage people to move away from the PUA in order to get texts that are cleaner and more in line with the Unicode model. But the only software that handles all OT features and works with all scripts/languages (including historic ones) is XeTeX. It produces beautiful type (I have set books with it) but it is so different from what people are used to, and the learning curve is so steep, that it's hard to recommend. Everything else out there has significant limitations -- very expensive, won't handle Plane 1 characters, only supports a limited set of OT features, etc.
Now we have LO 5.3: affordable, supports all the OT features I've tried even with Plane 1 characters, and is cross-platform. To someone like me who has experimented with various pieces of software and found them all lacking in one way or another (except XeTeX), this is a true gift. I understand that users accustomed to checkboxes/pulldowns/whatever to access OT features may think it's old-fashioned or inconvenient to learn and type codes. (Please don't say it's hard, because it isn't.) But this really is minor compared to not having access to the features at all.
That said, I would be happy help in any way I can with developing a better interface.