Public Lab Research note


Continuing difficulties with infrablue in cheaper CMOS cameras

by warren | June 18, 2013 15:35 | 4,279 views | 10 comments | #8261 | 4,279 views | 10 comments | #8261 18 Jun 15:35

Read more: unstable.publiclab.org/n/8261


I'm in charge of sourcing more compact and cheaper cameras for the Infragram Point & Shoot, and have been experiencing trouble with the images I'm able to take with modified cheaper cameras. It seems like the blue channel is leaking infrared -- something we haven't seen in the Canon cameras we've been converting. I think we'll figure out a way around this but wanted to share my results so far.

My attempt and results

Here is an image taken with the most recent (otherwise very promising!) sample camera:

9077620142_60d6ecd2ec_o.jpg

It looks OK, but when you look at each color channel (see lead image, at top), you notice that vegetation shows up white in the blue channel, where it's supposed to be absorbing (dark).

Also see that when you upload it to http://infrapix.pvos.org (the current Infragram software prototype) you get not very good differentiation in the generated NDVI, which makes sense if the blue channel is leaking (see actual images here):

Screen_Shot_2013-06-18_at_11.18.19_AM.png

I tried using the Curves function in Photoshop to identify a neutral grey, essentially post-white-balancing. That resulted in this more familiar looking image (better yellows):

post-color-balanced.jpg

Which, run through the Infragram software, wasn't much better:

Screen_Shot_2013-06-18_at_11.32.52_AM.png

Questions and next steps

Just to look at the differences, I ran up a couple "diff" images between channels. If there's enough differentiation, we can still do this with a modified Infrablue algorithm, because even with channel mixing, you can probably pick out the differences quantitatively (i.e. Blue+IR will still be different than Red+IR).

Here is (contrast stretched so you can see details, which is why it looks fuzzy) the Red channel minus the Blue channel:

Screen_Shot_2013-06-18_at_11.16.09_AM.png

And here is Red minus Green:

Screen_Shot_2013-06-18_at_11.24.21_AM.png

There are differences, but perhaps some more advanced subtraction/addition would be needed to pull out useful data, and that means the dynamic range of our resulting infrablue NDVI images will be much lower. I'm hoping we can figure out another solution. The issue seems to be mainly that the blue channel of the Bayer filter passes infrared, where in our Canons that is not true.

The options seem to be:

  • select a camera with a different Bayer filter which doesn't pass infrared
  • do some channel math to extract meaning from these images anyways
  • try to invent some sort of additional filter to get a clean blue channel...?

10 Comments

I think your analysis is good. NIR is passing through all three Bayer filters (even the green channel has light foliage). So there is not a very good visible-light only channel. I don't see how option 3 above (a filter to clean the blue channel) could work. NIR is passing through all Bayer filters and seriously contaminating all channels. You can't add a filter that blocks most of the NIR (you just removed one of those). If you add a filter that blocks some NIR, maybe the green channel would see mostly visible light and another channel would see mostly NIR. But it seems like a stretch.

Channel math might be the only option with this camera, but I never quite understood how that works.

Reply to this comment...


Re: option 3, i was half-heartedly thinking of making a second Bayer filter by photographing a checkerboard on film and printing lots of negatives, but it's really a bad idea, since you'd have to line it up quite well with the existing bayer filter upon installation, which is totally impractical.

Reply to this comment...


The idea of using green for visible light isn't nearly as good as blue or red... but Chris is right, the green channel doesn't seem to leak as much NIR, so I thought a little more about it.

Look at this image, from the old near-infrared-camera page (maybe we should resuscitate it, the graph is quite useful):

The green is certainly not as absorbed (the example is a graph of eucalyptus leaf absorption i found somewhere) but it is pretty close to red/blue, when you compare it to infrared. Granted, the sensor itself may be less sensitive to infrared -- so the graph we'd see of what the sensor actually detects may not be as differentiated. But green still seems to be quite correlated with photosynthetic absorption, and could still be quite useful.

See the diff between R and G channels, above -- although it's rather "grungy" data since the dynamic range wasn't good, but with more rigorous testing, perhaps it is useful... what would be the next step, testing this technique against the Canon NDVIs?

Is this a question? Click here to post it to the Questions page.

Reply to this comment...


Is there any thought or possibility to make this camera active rather than passive?

Is this a question? Click here to post it to the Questions page.

Reply to this comment...


From what I understand (which is not much, I'm pretty new to this) and have found online is that for some reason, CMOS cameras produce results that are not nearly as good as CCD digital cameras.

You can check out this site page which shows a few comparisons between CMOS and CCD cameras.
http://flightriot.com/category/vegetation-management/

Reply to this comment...


Yeah, that guy's been on the infrared mailing list a lot recently (http://publiclab.org/wiki/mailing-lists) and it's great to see his results. Still, I've seen some webcams (presumably CMOS) which do very well, like Stewart's --

http://publiclab.org/notes/gonzoearth/06-20-2013/rocketfish-hd-webcam-conversion

http://publiclab.org/notes/gonzoearth/07-02-2013/infrablue-webcam-panoramas-in-the-oakland-hills

I'm still trying to figure out what the difference is.

Reply to this comment...


Perhaps the exposure settings play a big part as well. Brenden at flightriot.com and myself (mostly him) have been experimenting with different camera settings. I am very new to this all and still learning but it seems that the White Balance settings play a huge role in the overall quality. Obviously the LUT is customizable and I believe that the processing end (imageJ/Fiji, ned horning plugin, etc.) are pretty straight forward (basically some addition/subtraction) so it has to be the exposure settings...right?

Is this a question? Click here to post it to the Questions page.

Reply to this comment...


Well, definitely WB is an element, but what seems to be the big thing is that on these cheaper cameras (with the exception of the one Stewart tested) the blue channel detects infrared light -- which is not true on the CCD and more expensive CMOS cameras we've modified.

Reply to this comment...


Is that something that could be corrected with a specific filter or an inherent issue with the quality of the camera?

Is this a question? Click here to post it to the Questions page.

Reply to this comment...


My best hypothesis right now is that it's related to the Bayer filter in the camera itself, so I'm hoping to order more different cameras and find one that works.

Reply to this comment...


Login to comment.