Time to go third party.
There are several, mostly similar, options for you. All you need is a 3rd party charger that can charge Olympus BLS-5 and/or BLS-50 batteries (see the related question, Does the Olympus OM-D E-M10 II use the same battery as the previous version?).
You could get a 3rd party charger for Olympus BLS-5/BLS-50 batteries that uses USB for power; however, USB tends to charge the batteries very slowly. USB will provide only 5V, but the batteries need 8.4V to charge. Such a charger will use a DC-DC boost inverter, but in doing so, the output current will necessarily be lower than the input current. Lower current supplied to the batteries translates into longer time to charge.
Since you are already comfortable sorting out the phones with a cigarette charger, I recommend also getting a battery charger with a cigarette lighter plug option. This will provide plenty of power to charge your batteries faster than via USB.
Here's an example 3rd party charger that has a built-in AC plug (non-polarized US plug, NEMA 1-15p), with a 12V DC input jack and corresponding 12V cigarette lighter cord, for $10: Wasabi Power Battery Charger for Olympus BLS-50, BLS-50, ... (Amazon.com US).
Note there others to be found, but they're all rebranded versions of the same thing (and so is the Wasabi, it was just an example). Many times they will come with 1 or 2 extra batteries. These batteries tend to be usable, but usually don't last as long on a single charge as the manufacturer-branded batteries.