The setting applies to the in-camera conversion from raw to JPEG. This means it is at an advantage vs. saving as a full-color JPEG, loading that in Photoshop, and adjusting. That's because the color choices like white balance and other tone curves will limit what you can do in conversion — maybe not hugely, but I think still worth considering. Additionally, you'll have to re-save after editing, which can introduce further compression artifacts.
If you shoot raw, the file may get a B&W conversion attached as information, but the color channels will be untouched. Some raw converters may take that as a starting point.
There is one other difference, though — many in-camera B&W modes are carefully tuned to provide a nice tone curve. This is like using someone's fancy B&W preset in a raw converter; in this case Olympus's preset. That's less flexible, but a lot of research and care by experts has gone into making these modes generally quite nice. So, maybe you'll get results you like with little effort — a quick search will find many blog reviews like this one where the reviewer is taken with the results from the Pen-F in specific.
And, actually, Olympus has gone to some lengths to make this tunable to your preferences. You can adjust a "shading effect" from -5 to +5, choose color filters in three strengths from across the spectrum, add film-grain simulation of various amounts (or not), and adjust shadow/highlight preferences too. So, you can create different "black and white films" that match your personal preference. This is far less flexible on a per-image basis than at-the-computer conversion, but it has its own appeal and advantages.
You can even shoot RAW+JPEG with one of these presets, so you then have the straight-out-of-camera image plus the raw data to play with if you get an image you want to tinker with. (There's really no wrong way to do this, but that's my personal recommendation.)
Chris Novak's point in another answer is also important: if you use this mode, you can actually have the EVF display in black and white, so you can see what you're going to get as you are shooting.