Prepare for an immense post - yes, this got out of hand...
Obligatory xkcd:

Unfortunately, there is no simple 'best' format. Some are very well supported, some offer extreme versatility, some offer lossless compression,...
The first part of this answer ("Features" & "Brief overview of the formats") will talk about technicalities, while the second part ("(Other) Things to consider") is more directed at the practical aspects of the choice of format.
Features:
Please note that it is almost impossible to include every hack to every format - e.g. GIFs can be saved without compression by ignoring the LZW table. Why do I not mention this below? Because 99% of all GIFs I ever encountered used LZW, because LZW today is a no-brainer in computation power, and because this post tries to clarify the situation for popular situations, not for ILM's R&D department. Photographers will use their files for archival, publishing and print, so these are the things I consider here.
Information cross-checked between the respective Wikipedia articles, specifications, Wiki's comparison, and exiftool's metadata-support-list.
| Bits per | | Supported by
Codec | Lossy | Channel | Metadata | Channels | Programs | Good for (IMHO)
-------------------------------------------------------------------------------------------------
BMP | n | <= 8 | - | RGBA | Most propr. & free | Archival
BPG | y | <= 14 | EXIF+XMP | RGBA | |
EXR | o | <= 32 | y(?) | RGBAD | | VFX workflow
FLIF | o* | <= 16 | EXIF+XMP | RGBA | | To be seen
GIF | n | <= 8* | XMP | RGB | Most propr. & free | GIFs ;-)
HEIF | o* | <= 16 | EXIF+XMP | RGB(A/D) | | To be seen
JPEG | y* | <= 8 | EXIF+IPTC+XMP | RGB | ~ all propr. & free | Online; Easy access
JP2K | o | <= 32 | EXIF+IPTC+XMP | RGBA | |
JXR | o | <= 32 | EXIF+IPTC+XMP | RGBA | |
PNG | n | <= 16 | EXIF+IPTC+XMP*| RGBA | Most propr. & free | CAD-drawings; Online
TGA | n | <= 8 | y(?) | RGBA | |
TIFF | o | <= 32 | EXIF+XMP | RGBA | Most propr. & free | Archival; Editing
WebP | o | <= 8 | EXIF+XMP | RGBA | |
Legend: o
... Optional; n
... not available; y
... available; D
... Depth; *
...Look below at according text.
Brief overview of the formats:
BMP
Feature |
-----------------------------------------------------------------
Introduced | 1990
Open + Free | Both per Microsoft's Open Specification Promise
Colorspace | R:G:B[:A] (4:4:4[:4])
b/c/p | 1:0:0[:0], 5:6:5, 8:8:8[:8]
Compression | None [RLE in 5:6:4] (so: lossless)
Maximum Size | 4 GiB
Metadata | [ICC]
OS support | Virtually all OSs with a graphical interface
Legend: b/c/p
... bits per channel (e.g. R,G,B) per pixel. things in [ ]
are optional; ?
...educated guess / no clue.
'Bitmap' files are encoded in lines and are usually not compressed, so a single bit flip will only destroy one line of the image As long as it does not flip the header, which will make decoding harder - try it for yourself with a HEX editor!. Since it does not offer (good) compression, file sizes are huge, as it has to save the full information for each pixel. Because of its rigidity, it may be good for long-term archival.
BPG
Feature |
---------------------------------------------------------------------
Introduced | 2014
Open + Free | Yes (but HEVC patents might be problematic)
Colorspace | R:G:B[:A] (4:4:4[:4]); Y:Cb:CR[:A] (4:2:0[:4] - 4:4:4[:4]);
| Y:Cg:Co[:A] (4:2:0[:4] - 4:4:4[:4]); C:M:Y:K (4:4:4:4)
b/c/p | 8 - 14
Compression | HEVC (lossy / lossless)
Maximum Size | ?
Metadata | [EXIF]; [ICC]; [XMP]
OS support | Linux, Mac, Windows (at least through browser decoding)
Legend: b/c/p
... bits per channel (e.g. R,G,B) per pixel. things in [ ]
are optional; ?
...educated guess / no clue.
'Better Portable Graphics' (BPG) uses HEVC, which you may know from the h.265 video codec. It was meant to be the successor to JPEG, but never got popular enough. With the rise of HEIF, which is quite similar in some ways but more popular, it is plausible that HEIF will be preferred. HEVC is far superior in terms of compression compared to JPEG's DCT - however, it does not compare well in all but the lower bit rates, as it tends to be blurry.
EXR
Feature |
---------------------------------------------------------------------
Introduced | 1999
Open + Free | Yes
Colorspace | R:G:B[:A][:D] (4:4:4[:4][:4])
b/c/p | <= 32
Compression | [RLE]; [ZIP]; [PIZ]; ... [lossless (usual) / lossy]
Maximum Size | > 4 GiB
Metadata | [Yes (XMP-style)]
OS support | Linux, Mac, Windows (through library)
Legend: b/c/p
... bits per channel (e.g. R,G,B) per pixel. things in [ ]
are optional; ?
...educated guess / no clue.
OpenEXR was designed by Industrial Lights and Magic (ILM) as an intermediate format for VFX workflows. It can hold multiple channels at very high bit depths, multiple images, and metadata in one file. It offers different compression algorithms - or no compression at all. EXR can be compared with TIFF - EXR offers more options, while TIFF is far mroe popular.
FLIF
Feature |
---------------------------------------------------------------------
Introduced | 2015
Open + Free | Yes
Colorspace | R:G:B[:A] (4:4:4[:4]) (CMYK and YCbCr in ToDo-List)
b/c/p | <= 16
Compression | MANIAC (variant of CABAC, used in AVC/HEVC) (lossless / lossy (1st generation))
Maximum Size | > 4 GiB
Metadata | [EXIF]; [ICC]; [XMP]
OS support | Linux, Mac, Windows (through provided viewer)
Legend: b/c/p
... bits per channel (e.g. R,G,B) per pixel. things in [ ]
are optional; ?
...educated guess / no clue.
'Free Lossless Image Format' (FLIF) uses a derivate of HEVC compression that is lossless. FLIF claims to have extreme compression ratios in comparison to all other formats of the time - while my own tests led me to believe this, it really needs computing power to be usable (Several minutes of encoding time for a single 24 MP picture with a hyperthreaded 4,3 GHz hexacore is not that good :D). However, as it is a young codec, improvements might come forth. It offers support for animations, alpha channels, progressive decoding, and even lossy encoding (with no more generation loss after the first encoding). Only time will show if it will succeed, and to be honest, I quite hope so, as it seems to offer a single solution for multiple problems.
GIF
Feature |
---------------------------------------------------------------------
Introduced | 1987
Open + Free | Yes
Colorspace | R:G:B[:A] (4:4:4[:4])
b/c/p | 2 (palette of 256 colors in total)
Compression | LZW (lossless)
Maximum Size | < 4 GiB
Metadata | [XMP]
OS support | Virtually all OSs with a graphical interface
Legend: b/c/p
... bits per channel (e.g. R,G,B) per pixel. things in [ ]
are optional; ?
...educated guess / no clue.
While 'Graphics Interchange Format' (GIF) offers 8 bits per channel per pixel, it will reduce them to a color palette of 256 colors (which can include a "background color"). It is mostly used for animations - the only thing that PNG cannot do better, as PNG in itself does not offer animation support.
HEIF
Feature |
----------------------------------------------------------------------
Introduced | 2015
Open + Free | No (patents)
Colorspace | ? Y:Cb:Cr[:A/:D] (4:2:0[:4]) ?
b/c/p | <= 16
Compression | HEVC (lossy)
Maximum Size | < 4 GiB
Metadata | [EXIF]; [XMP]
OS support | Linux, Mac, Windows
Legend: b/c/p
... bits per channel (e.g. R,G,B) per pixel. things in [ ]
are optional; ?
...educated guess / no clue.
'High Efficiency Image Format' (HEIF) uses HEVC for compression, too. In addition to the color channels, it can also hold either an alpha channel or a depth map (used for later software depth-of-field effects). Also, rudimentary editing can happen losslessly. Accoding to specs, it also has a lossless compression mode. Since all major OSs support it, it seems like the most likely contender for a succession of JPEG (if there ever is one).
JPEG
Feature |
----------------------------------------------------------------------
Introduced | 1991
Open + Free | Sort of (free library, but patent might apply)
Colorspace | Y:Cb:Cr (4:2:0 (typical) - 4:4:4)
b/c/p | 8
Compression | DCT (lossy)
Maximum Size | < 2 GiB
Metadata | [EXIF]; [ICC]; [IPTC]; [XMP]
OS support | Virtually all OSs with a graphical interface
Legend: b/c/p
... bits per channel (e.g. R,G,B) per pixel. things in [ ]
are optional; ?
...educated guess / no clue.
'Joint Photographic Experts Group' (JPEG) is arguably the most used image format around today. It uses the discrete cosine transformation (DCT), which is of the lossy kind.There is a lossless specification, but it is not used too often. Certain programs can perform certain rudimentary actions (e.g. rotation) losslessly, though this also requires the image width and height to be divisible by 8 (the block size of JPEG) - e.g. 800x640 will work, 804x643 will not. JPEG has no option to save images in RGB - it transforms the picture into the YCbCr colorspace and often reduces the pixel information from 4:4:4 (every pixel has all channels) to 4:2:0 (every channel has luminance, but only every 4th pixel gets a Cb/Cr-value). As with most colorspace conversions, this can lead to perceivable differences especially in extreme colors. JPEG is quick to encode and not too bad in high quality settings, but to me, the things mentioned above would not make me weep if it ever vanished - it served us well, but the used image formats could be a bit more...recent. After all, computers evolved well since 1991.
JP2k
Feature |
----------------------------------------------------------------------
Introduced | 2000 (duh...)
Open + Free | No (patents)
Colorspace | ? Y:Cb:Cr[:A] (4:4:4[:4]) ?
b/c/p | 8 - 32
Compression | Wavelet (lossy / lossless)
Maximum Size | ?
Metadata | [EXIF]; [ICC]; [IPTC]; [XMP]
OS support | Linux, Mac, Windows (at least through viewer programs)
Legend: b/c/p
... bits per channel (e.g. R,G,B) per pixel. things in [ ]
are optional; ?
...educated guess / no clue.
'JPEG 2000' (JP2k or JP2) is the official successor to JPEG. It uses wavelets instead of the DCT, which offer less blocky artefacts and is overall more versatile than JPEG. Despite all this, it never really caught up with JPEG.
JXR
Feature |
----------------------------------------------------------------------
Introduced | 2009
Open + Free | Yes (Microsoft Open Specification Promise)
Colorspace | Y:Cb:Cr[:A] (4:2:0[:4] - 4:4:4[:4]); Y:Cg:Co[:A] (? 4:2:0[:4] - 4:4:4[:4] ?);
| C:M:Y:K [4:4:4:4]
b/c/p | 8 - 32 (16 for CMYK)
Compression | DCT (lossy / lossless)
Maximum Size | ?
Metadata | [EXIF]; [ICC]; [IPTC]; [XMP]
OS support | Linux, Mac, Windows (at least through viewer programs)
Legend: b/c/p
... bits per channel (e.g. R,G,B) per pixel. things in [ ]
are optional; ?
...educated guess / no clue.
'JPEG extended range' (JPEG XR, JXR) is another try to succeed JPEG. Its YCgCo colorspace is superior to YCbCr because it is entirely reversible. While some software supports it, it too never got close to the fame of other formats.
PNG
Feature |
----------------------------------------------------------------------
Introduced | 1996
Open + Free | Yes
Colorspace | R:G:B[:A] (4:4:4[:4])
b/c/p | 8 - 16
Compression | DEFLATE (lossless)
Maximum Size | ?
Metadata | [EXIF]; [ICC]; [IPTC]; [XMP]
OS support | Virtually all OSs with a graphical interface
Legend: b/c/p
... bits per channel (e.g. R,G,B) per pixel. things in [ ]
are optional; ?
...educated guess / no clue.
'Portable Network Graphics' (PNG) was introduced as a successor to GIF. While it is lossless by design, PNG files can be optimized with several tools, some of which will compress the file in a lossy way. PNG uses the DEFLATE compression, so it is quite efficient for graphics (like CAD drawings, screenshots,...), but less efficient for photographs. While it offers support for metadata, some programs have trouble reading them. Thanks for the heads-up, @mattdm!
TGA
Feature |
----------------------------------------------------------------------
Introduced | 1984
Open + Free | ? Yes
Colorspace | R:G:B[:A] (4:4:4[:4])
b/c/p | <= 8
Compression | RLE (lossless)
Maximum Size | ? < 2 GiB
Metadata | Rudimentary
OS support | ? Virtually all OSs with a graphical interface
Legend: b/c/p
... bits per channel (e.g. R,G,B) per pixel. things in [ ]
are optional; ?
...educated guess / no clue.
'Truevision TGA' / 'TARGA' (TGA) is a fie format that I only included because everyone seems to know it. It was introduced in 1984. It supports lossless compression (RLE) which will work okay for graphics, but not-so-well for photographs.
TIFF
Feature |
----------------------------------------------------------------------
Introduced | 1986
Open + Free | ? Yes
Colorspace | R:G:B[:A] (4:4:4[:4]); Y:Cb:Cr[:A] (? 4:2:0[:4] - 4:4:4[:4] ?);
| C:M:Y:K (? 4:4:4:4 ?); L:a:b[:A] (? 4:4:4:[A] ?)
b/c/p | 8 - 32
Compression | [LZW (lossless)]; [ZIP (lossless)]; [JPEG (lossy)]
Maximum Size | ?
Metadata | [EXIF]; [ICC]; [XMP]
OS support | Virtually all OSs with a GUI support >= 1 of the compression types
Legend: b/c/p
... bits per channel (e.g. R,G,B) per pixel. things in [ ]
are optional; ?
...educated guess / no clue.
'Tagged Image File Format' (TIF or TIF) has been around for a long time, too. It offers layer support (i.e. multiple RGBA-images stacked). TIFFs are often used as intermediate files because the are widely supported and quite flexible in terms of their capabilities.
WebP
Feature |
----------------------------------------------------------------------
Introduced | 2010
Open + Free | Yes
Colorspace | R:G:B:A (4:4:4[:4]) lossless; Y:Cb:Cr[:A] (4:2:0[:4]) lossy
b/c/p | 8
Compression | VP8 (lossless / lossy)
Maximum Size | ?
Metadata | [EXIF]; [ICC]; [XMP]
OS support | Linux, Mac, Windows (at least through browser decoding)
Legend: b/c/p
... bits per channel (e.g. R,G,B) per pixel. things in [ ]
are optional; ?
...educated guess / no clue.
'WebP' uses VP8 (an open source rival format to AVC). As with BPG, it never made the leap into consumer devices, though it seems that it is used by many internet services.
(Other) Things to consider:
Re-encoding (generation loss)
Re-encoding a lossless file will not alter anything - re-encoding a lossy file will almost certainly lead to artefacts. JPEG can handle this pretty well if you save the file in the same quality setting that it was saved at before.
This video shows generation loss pretty well - the first frame shows the original file, while all others show re-compression at different quality settings. (Note that FLIF is in lossy mode, so the first frame will look different.)
Artefacts will not necessarily be a death sentence - e.g. for quick web publishing or preview on mobile devices, it might not be too bad.
Longevity of codec
When writing this answer, I was thinking to myself "who would use TARGA nowadays, anyway?" and it made me think: I would not ever hesitate to drive a car made in the 80's. I would not hesitate to look at pictures shot in the 80's. I would use any cameras made in that time. But I would not use a codec that old. Why?
In the end, there is no sure way to say whether one codec or the other will survive a certain time span. If HEIF were to replace JPEG on all consumer devices tomorrow, how long would it take for programs to cease JPEG support? How many generations of computers - and more importantly: OSs - will there be before you can no longer open them?
On the other hand, relatively simple codecs like TARGA only demand for relatively simple programs to read them, while modern codecs and their decoders have multiple dependencies. So while simplicity is bad for compression, it might be good for archival in an apocalyptic scenario. Thanks @lijat for pointing this out!
In my opinion, this needs several angles to consider: Which codec is popular enough so that support will not drop immediately? Which codec is supported by the open source community (because noone will maintain proprietary formats from a bankrupt company)? Also, it seems that at least every decade or so, one should see if there is a need to jump to a new, better supported codec (See "Re-encoding (generation loss)") - you would not, for example, want your TARGA-collection to be unreadable tomorrow, right?
That, by the way, is especially worrying when thinking about RAW files.
Program support (Longevity #2)
The most popular, best codec will not be good enough if you cannot use it. And while I would not use inferior codecs just because a particular program does not support it, it might be bad to use a codec that just one program properly supports.
What features do I need?
Personally, I still encode most of my files in JPEG - I can read them on any device and I can barely (if at all) see the artefacts. 8bit is good enough for most devices and alpha channels are not really needed when just viewing pictures.
For all files that are not "edit once"-style, I either keep my RAWs or at least 16bit TIFFs so that they are still useable in the future.
PSD? DNG?
"Photoshop Document" (PSD) is Photoshop's TIFF-styled format. Technically, it is quite similar to TIF. There is PSB, too, which is the same thing just for file sizes over 4 GiB. There is nothing wrong with using it, but personally, I prefer TIFF as far as possible.
"Digital Negative" (DNG) is an attempt to create an open RAW standard. While I love the idea and it works quite well, note that some RAW editors have trouble with them - e.g. Capture One typically forgets the camera's white balance, thus setting the slider to 5000K, no matter what the actual value is. Other programs in the past have shown them as solid white or pink images or give them a magenta hue. If file size is of no concern to you, then you can include the original RAW in your DNG - if you ever need it again, you can simply extract it again. My 2 cents? Try it out with your favourite software - and if it works well, use it.
Other formats?
Since this already got out of hand, I did not want to address even more image formats. However, this does not mean that those not listed are not worth considering.