New Sensor for Elphel 373

Rod Supports, the Dictator Interface, Viewfinder Tablet, everything related to hardware goes in here.
124 postsPage 10 of 131 ... 7, 8, 9, 10, 11, 12, 13

Re: New Sensor for Elphel 373

Postby shijan » Tue Apr 24, 2012 1:11 am

CMV2000 HIGH DYNAMIC RANGE MODES:
The sensor has different ways to achieve high optical dynamic range in the grabbed image:
- Interleaved read-out: the odd and even rows have a different exposure time (i believe its same as pixel binningo)
- Piecewise linear response: pixels respond to light with a piecewise linear response curve. (quote from developers: this mode causes problems for color reconstruction and color correction because it operates in non-linear response). and I personally can't understand what is it means
- Multi-frame readout: Different frames are read-out with increasing exposure time (i believe its a classic hdr mode with double exposed frames which must be blended in post)
shijan offline
Posts: 85
Joined: Thu Nov 04, 2010 4:05 am

Re: New Sensor for Elphel 373

Postby tequilamango » Tue Apr 24, 2012 1:36 am

" - Interleaved read-out: the odd and even rows have a different exposure time (i believe its same as pixel binningo) "

I think this is different, because BINNING is the clustering of more than one pixel to act like one, it is used to reduce the read noise but on the sensor resolution expense.
tequilamango offline
Posts: 101
Joined: Wed Feb 08, 2012 4:15 pm

Re: New Sensor for Elphel 373

Postby Sebastian » Tue Apr 24, 2012 7:30 am

I think this is different, because BINNING is the clustering of more than one pixel to act like one, it is used to reduce the read noise but on the sensor resolution expense.


Correct, it has nothing to do with binning, the sensor does not combine or average any photosites, its just an image with rows having different exposure times. Also note that this mode acts differently with monochrome and bayer pattern sensors.
Monochrome: even/odd rows are affected
Bayer Pattern: one 2x2 photosite GRBG block has same exposure time so its actually alternating exposures every 2 lines.

I got an update from cmosis yesterday, good news:
90dB = 15 F-stops is in piecewise linear response mode, I don't see any problem with color reconstruction, you just need to be aware of the changed response curve, but we define the curve ourselves very precisely so we can easily store it with the footage and unapply the curve and transform pixels back into linear color space at a higher bitdepth if we want to in post. Its similar to log modes in other cinema cameras.

In Interleaved read-out you can use one knee instead of the 2 and cmosis measured the max DR in that mode with 80db = 13.3 F-stops.
Sebastian offline
Posts: 1938
Joined: Wed Nov 03, 2010 6:25 pm

Re: New Sensor for Elphel 373

Postby shijan » Tue Apr 24, 2012 9:41 am

Correct, it has nothing to do with binning, the sensor does not combine or average any photosites, its just an image with rows having different exposure times. Also note that this mode acts differently with monochrome and bayer pattern sensors.
Monochrome: even/odd rows are affected
Bayer Pattern: one 2x2 photosite GRBG block has same exposure time so its actually alternating exposures every 2 lines.
So in the end you got a picture with light and dark pixel stripes? its really strange mode) maybe for some technical applications

I got an update from cmosis yesterday, good news:
90dB = 15 F-stops is in piecewise linear response mode, I don't see any problem with color reconstruction, you just need to be aware of the changed response curve, but we define the curve ourselves very precisely so we can easily store it with the footage and unapply the curve and transform pixels back into linear color space at a higher bitdepth if we want to in post. Its similar to log modes in other cinema cameras.
so they called a some kind of log sensor curve for sensor operation - non linear response mode?
shijan offline
Posts: 85
Joined: Thu Nov 04, 2010 4:05 am

Re: New Sensor for Elphel 373

Postby Sebastian » Tue Apr 24, 2012 10:49 am

So in the end you got a picture with light and dark pixel stripes? its really strange mode) maybe for some technical applications


I agree you cannot "use" it directly without post-processing, it would require either a debayering algorithm that is aware of the 2 different strips and can gather gather properly from the correct exposure or a tool that splits the image in 2 different full exposure images trying to reconstruct the missing strips from the other exposure image. Remember we have twice the amount of rows than FullHD/2K so if the target is a downsized 2K image this could work very well.

so they called a some kind of log sensor curve for sensor operation - non linear response mode?

Its an approximation of a log curve and its called piecewise linear response curve.
Here I created an Illustration:
DR-illustration-01.jpg
Sebastian offline
Posts: 1938
Joined: Wed Nov 03, 2010 6:25 pm

Re: New Sensor for Elphel 373

Postby SimonL » Tue Apr 24, 2012 10:51 am

This old PDF from 2002 has some words about color. The explained effect is due to the Bayer pattern, two adjacent R, G or B filtered pixels used for the calculation of one final RGB pixel may not result from the same response curve.
There are some ways to tweak and correct this during "non-global non-linear response debayering" (hahaha).
As Sebastian said, there must be re-linearization of the data in post.

Also I don't think 15 f-stops are always required, well if that's what I have that's perfect but if I can do fine with 12 and avoid some heavy automated approximation in post, I'll go with it.
SimonL offline
Posts: 72
Joined: Mon Mar 12, 2012 5:22 pm
Location: Paris, France

Re: New Sensor for Elphel 373

Postby Guest » Tue Apr 24, 2012 6:13 pm

Sebastian wrote:The idea for the new sensor-front-end is to place an additional FPGA on the PCB that downscales the 12Megapixel images to something reasonable (like FullHD) and converts the data from into a format that the current 353 camera can work with. So the new sensor-front-end would would be sending image data the same way as the current sensor-front-end on all 353s does. That way its fully compatible with 353 and will also be fully compatible with 373.

When the performance of the image processing pipeline increases with new Elphel hardware (like 373) we can simply increase the resolution/FPS of the data that our FPGA on the sensor-front-end generates with a software update.

So the new sensor-front-end would not allow us to capture 12 Megapixels at 300FPS, sorry ;)


I'm not fully getting what shoehorning the sensor into the 353 buy you. It sounds like you have to modify the case anyway and develop the FPGA support, and it would not even give you access to the full performance of the sensor as you point out.

Once you have an FPGA in the loop, why not have it ingest stream from the LVDS pair and stream of the raw footage over GigE? (GigE is by far the simplest, cheapest, and most available solution. USB 3.0 is not of these things).
Guest offline

Re: New Sensor for Elphel 373

Postby Sebastian » Tue Apr 24, 2012 6:29 pm

The quoted text is by now outdated. Latest announcement from just recently changed that plan a bit (in line with your concerns) http://apertus.org/en/apertus-camera-dp-team-up
Sebastian offline
Posts: 1938
Joined: Wed Nov 03, 2010 6:25 pm

Re: New Sensor for Elphel 373

Postby shijan » Tue Apr 24, 2012 7:56 pm

I agree you cannot "use" it directly without post-processing, it would require either a debayering algorithm that is aware of the 2 different strips and can gather gather properly from the correct exposure or a tool that splits the image in 2 different full exposure images trying to reconstruct the missing strips from the other exposure image. Remember we have twice the amount of rows than FullHD/2K so if the target is a downsized 2K image this could work very well

Afraid that such downscale with line slipping can cause aliasing.

And if to go with third method (Multi-frame readout) combined with Piecewise linear response - it will work in same way as HDRx from RED.

BTW 13.3 f-stops its really decent latitude too (same as at Sony f3 and Blackmagic camera) so if there is any advantage to use those one knee mode it can be a good compromise between simplicity vs dynamic range
shijan offline
Posts: 85
Joined: Thu Nov 04, 2010 4:05 am

Re: New Sensor for Elphel 373

Postby Guest » Tue Apr 24, 2012 10:21 pm

Sebastian wrote:The quoted text is by now outdated. Latest announcement from just recently changed that plan a bit (in line with your concerns) http://apertus.org/en/apertus-camera-dp-team-up


Ah, right, thanks. How open will the camera be? Will I be able to change the FPGA design (I think it's safe to assume there will be one)? Will the sensor datasheet be available to us? This may require special negotiation with CMOSIS, but seems rather crucial to the goal of the project. Do we have any raw samples from this sensor already?

The digital Bolex and BlackMagic's 3K camera really upped the ante here. The "A lot less than $10k" doesn't say much, but I don't think this is viable post $5-6k.

Thanks
Guest offline

PreviousNext
124 postsPage 10 of 131 ... 7, 8, 9, 10, 11, 12, 13

Who is online

Users browsing this forum: No registered users and 1 guest

cron