3
\$\begingroup\$

I have recently discovered 5 instances of images where a single pixel row is discolored, kind of what you experience with hot pixels. What I find weird about it is that if I look at the position of the row, it is not the same row in any instances, like you would experience with hot pixels. Also, it seems to happen at random. I can't really draw any conclusions based on only 5 instances, but here are the facts I have been able to gather so far:

  • All images were taken with a Canon EOS R, with the latest firmware (v.1.8)
  • All images are shot in RAW (.CR3)
  • One image were taken with a Tamron 150-600mm VC G2
  • Four images were taken with a Tamron 70-300mm VC
  • A couple of EF and RF lenses were used in the same period, but without incidents.
  • Both lenses and camera has been used together for a couple of years without incidents, then the first instance occurred on July 14th 2021, one on the 25th and then 3 on the 28th.
  • The occurrences happened on 3 different SD cards, although all were SanDisk Ultra 32GB.
  • In some cases I have been tracking an animal for a while, in others more or less just brought the camera to my eye, composed and focused and the taken a single shot.
  • The five instances are out of approximately 2400 images. In some cases I have 4 or 5 images taken within a second (according to EXIF), and it just happens at random at one of the images in the burst.
  • The line does not always have the same colored, but is mostly magenta. It seems to change colored depending on the colored it should have had.
  • The line is sometimes only around a quarter of the image width, in other almost all across the image.
  • The lines are visible when viewing the RAW file directly from the SD card on my PC, so it isn't something that happens during import.

hot pixel line example 1

hot pixel line example 2

Please notice that these are crops of the affected areas, they are not cropeed at the same size and the lines are not at the same position in the images.

As it is not the exact same pixel row every time, I suspect one of these causes:

  • Buffer corruption. (either internal memory or when writing to SD card)
  • Sensor overheating.
  • Firmware problem (as first occurrence were after the upgrade to firmware v.1.8)
  • Bad SD cards (although it seems unlikely to happen to 3 different cards within 14 days)
  • Lens compatibility (which I also doubt, as I have used both lenses with this camera for 2 years without incidents)

I did notice that when turning on the camera with the lens hood on, that a couple of hot pixels were visible on the display. I then took of the lens, put on the mount cap and did a sensor clean without a lens mounted, which makes the camera map out hot pixels. As expected, the image on the display were perfectly black afterwards. This MAY be a factor, but remains to be proven.

Does anyone have a qualified guess as to what could have caused these issues?

\$\endgroup\$
9
  • \$\begingroup\$ Can you provide the raw files (on a file sharing service)? If you were shooting raw+jpg, can you also provide the original jpgs? \$\endgroup\$
    – xiota
    Jul 29, 2021 at 17:12
  • \$\begingroup\$ Of course. I have a file with a very long line. Nearly missed it as it were added to my "to-be-delete" batch. It can be downloaded from drive.google.com/file/d/1fxHMcbvjXQcPvF2ASSzfKucfPe9um87_/… \$\endgroup\$ Jul 29, 2021 at 17:22
  • \$\begingroup\$ Puzzling... I doubt the lens would cause it unless it's some processing related to changing lenses, like lens correction. Since you state that the line moves around, it's probably not the sensor itself, but some connection or processing between the sensor and memory card. Is it possible to downgrade the firmware? You may need to ask Canon for their opinion. \$\endgroup\$
    – xiota
    Jul 30, 2021 at 2:58
  • \$\begingroup\$ No. Firmware downgrade is not possible. Perhaps it is an issue with reading sensor output while something else goes wrong.. e.g. communication with lens fails, mapping out hot pixels on the sensor or similar. I will keep an eye on it, so far it has never happened with a Canon lens, if it does I will get in touch with Canon to hear what they think the cause might be. \$\endgroup\$ Jul 30, 2021 at 7:35
  • \$\begingroup\$ I'm gonna take a broad swing at "data transmission error" It's a bit like when a computer's SATA cable is getting flaky. 99.9% of the time everything is fine, then something will look at it the wrong way & you get bad read/writes. Testing the drive gives random errors, or no errors… until the owner starts replacing parts at random. Then they change the SATA cable & spray the contacts & all is well again. I'd guess in a camera any similar connectivity issue [or even a dying capacitor could behave similarly] is going to be a nightmare to accurately diagnose… \$\endgroup\$
    – Tetsujin
    Jul 30, 2021 at 8:29

3 Answers 3

2
\$\begingroup\$

Summary:

It appears that the corruption consists of periods of low or no blue channel signal.

The examples below show very low blue channel output even when the corruped image appears somewhat blue.

As my crops may include small amounts of adjacent uncorrupted image the blue channel may be fully dead when corrupted.

This seems highly likely to be a camera related error.


The "why" is uncertain, but taking a histogram of various parts several of several of the images shows in the corrupted lines the blue channel is very low in each case.

I'd expect the problem to be in the camera rather than lens or memory card.
(I may be wrong). Even in your "animal ear" crop where the defective line looks very blue it still has low blue channel

enter image description here

In the photo you posted to web the low blue channel in the corrupr area and quite different result on the adjacent uncorrupted wall can be seen here.

enter image description here

\$\endgroup\$
2
  • \$\begingroup\$ Although I agree that a camera fault is most likely, I can’t figure out what your evaluation of the histograms are meant to imply? \$\endgroup\$ Jul 31, 2021 at 13:40
  • \$\begingroup\$ @JohnSørensen See addition to answer \$\endgroup\$ Jul 31, 2021 at 21:23
2
\$\begingroup\$

Unfortunately, this looks like a serious defect in the camera's main board — most likely an out-of-spec row of RAM that is randomly losing its data because it is either being run at slightly too low a voltage or isn't being refreshed quickly enough.

The RAW data is getting corrupted on its way into the camera from the sensor. The reason I can be certain of that is because you're shooting in a compressed raw format, so you can see severe compression artifacts (ringing) above and below the bad row for several lines in each direction. The only way you'd get that sort of corruption would be if the data were damaged prior to compression. (Damage to the actual compressed data looks very, very different — like 8x8 square blocks that don't match what's above/below/beside them.)

Two things here:

  1. You really shouldn't shoot in compressed RAW format. If you were shooting in real RAW, this would probably result in only a single blown-out row instead of five, which you could fix with Photoshop or whatever.

  2. It's entirely possible that switching from compressed raw to true raw would eliminate the problem entirely by preventing the camera from using the chunk of RAM that is defective, but there's no way to know for sure without trying it.

Either way, if the camera is under warranty, I'd reach out to Canon, because this is almost guaranteed to be a hardware failure in the camera.

If the row were consistent, I'd blame the sensor, and even now, there's a possibility that it still be a defective amplifier or something, but my money would be on defective RAM on the camera's motherboard.

\$\endgroup\$
4
  • \$\begingroup\$ Valid points, and I think you may be right. Except for one thing. I did not shoot in compressed RAW. Never have. The RAW file I provided is ~34Mb, which seems to fit the chart under File Size and Media on this page: the-digital-picture.com/Reviews/Canon-EOS-R.aspx. Had it been C-RAW file size should have been significantly smaller. Regardless, I think there is a fairly good chance that your assessment is right. Unfortunately the camera is no longer under warranty. I will have to live with the periodic failures if they keep occurring.... \$\endgroup\$ Aug 1, 2021 at 7:38
  • \$\begingroup\$ As mentioned in the comments on my original question, I seem to have solved an issue with VC enabling/disabling by cleaning the lens mount contacts. Perhaps this could have resulted in short period of low voltage? \$\endgroup\$ Aug 1, 2021 at 7:40
  • \$\begingroup\$ Wow, I assumed that was DCT blooming, but if it isn't, then that means it's sensor bleed. Jeez. I've never see it isolated in a line like that. It's usually just a small area around a real (optical) hot spot. It might point to some of the pixels failing to get zeroed out and continuing to accumulate so much charge that they bleed into the next pixels, or maybe some software bug combined with a leaky shutter. Does it happen every time or just the first shot after you start using the camera after a while? \$\endgroup\$
    – dgatwood
    Aug 1, 2021 at 17:49
  • \$\begingroup\$ I have looked at ~2400 images taken since July 14th and found 6 instances, so the mean time between error must be ~400 shots. I have a burst series of shots, with 4-5 images taken within a second. Only 1 of the shots have this issue. In another case I have 3 shots with 1+ seconds interval, and only one is affected. The odd thing is that I have used this setup the same way for 2 years without occurrences. After upgrading to firmware 1.8 it has happened 6 times. I am still hoping that the VC issue I solved also solved this issue, but I will need to take more images before I can make sure. \$\endgroup\$ Aug 1, 2021 at 18:46
2
\$\begingroup\$

Well it can't be Sensor damage/pixel issues - This would result in the line being at the same position every time.

You can also definitely rule out the SD cards or transfer errors or bit flips. If that would be the case the image would get distorted way differently, like this (both images are a single bit "transfer error" right around where the issue starts with the line issue:

This is how an bit error would look like on the SD card Another sample of a single bit error

It always gets blocky. CR3 is a compressed format and even a single bit error will make wide parts of the image unusable even if not using C-RAW. CR2 does react a lot more gracefully to bit issues.

I would also rule out RAM issues if the line always cuts off at the end of the image, if it would be a RAM readout issue this would not discriminate to a single line but the line could also wrap around randomly (or actually not randomly, the image size with raw shots is the same very time, so the same shots should be affected).

My current guess would be amplification circuitry being flaky. If this skips a clock beat it might read out the rest of the line of pixels into the wrong color channel of the pixel pattern which might cause color degradation like that.

As for the hot pixels you see on the sensor, this is normal to see those when there is no light on the sensor. Mirrorless cameras will try to go above and beyond all reasonable ISO ratings to show you something that is not pitch black in the viewfinder, so it's just boosting the sensor readout until it can show you something, the hottest pixels on the sensor. This will be visible on probably all Canon EVILs - My M50 and R6 also show hot pixels when capped and looking into the EVF.

\$\endgroup\$
2
  • \$\begingroup\$ You're assuming that the RAM usage is a packed array. I could easily see them aligning each row to a page boundary to make address computation easier, depending on what they're doing with the data. Also, you're assuming that they use a fixed-address buffer for everything. I wouldn't assume that. I'm pretty sure DRYOS has normal memory allocation support like any other modern OS. \$\endgroup\$
    – dgatwood
    Aug 5, 2021 at 0:56
  • \$\begingroup\$ But even though sensor damage isn't directly causing this, isn't there still the option that "offloading" data from the sensor to memory could be faulty? I mean when a picture has been taken, the sensor information needs to be read and buffered in memory and from there written to a .cr3 file. Even if the sensor is fine and the SD card reader is fine and the SD card itself is fine, the process of reading sensor data could fail. \$\endgroup\$ Aug 5, 2021 at 8:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.