Actually there is a way to do exactly what you've described: quickly toggle to the original unedited photo using keyboard shortcuts and without leaving the Library module. It may not be as elegant as you were hoping for but here it is:
- Hit Ctrl+Shift+R (Cmd+Opt+R on a Mac) to reset all develop settings and revert to the original image.
- Hit Ctrl+Z (Cmd+Z) to undo that and recover your develop settings.
As I say, not the most elegant solution but it works fine. It's safe too: the reset action just gets added to your develop history, so if you forget to undo you can always come back and do it later.
Those commands are also available on the menu of course:
- Photo > Develop Settings > Reset
- Edit > Undo Reset Settings