Hi,
I'm trying to wrap my head around the rolling shutter effect, specifically why it happens.
I'm having a hard time understanding how the readout speed affects the image. If I understood correclty, when in electronic shutter mode the pixels are exposed as indicated by the shutter speed (e.g. at 1/1000 each pixel is exposed for 1/1000 of a second).
If the readout takes 1/100 s to scan the entire sensor, what happens exactly when I take the picture? Do the pixels start firing sequentially as the shutter speed dictates (i.e. 1/1000 s each, sequentially)? If that is the case, do they wait for the readout to catch up or do they continue firing? If the latter, by the time the readout reaches the second pixel, the eleventh pixel is firing, so there are 10 pixel between the one firing and the one being read. Does it work like this?
If the pixels are exposed for 1/1000 s and then turned off and their value stored, wouldn't that mean that the image should not be affected? I mean, they saw the subject for 1/1000 s and the motion should be frozen, they are just waiting for the value to be read. Just like if you asked 10 people to open their eyes for 1 second (shutter speed), one after the other, and draw what they see. They saw if for one second each, so at most the difference in the position of what they saw should cover 10 seconds. Then they can take hours to draw what they saw (readout speed), but what they saw specifically wouldn't be afftected by how long it takes them to draw it. Am I wrong here maybe?
Also, in general, why is mechanical shutter not as affected (if affected at all) by the rolling shutter effect? Does the sensor capture light differently when in mechanical shutter mode?
I just don't get it. I feel like I'm close to understanding why, but I still don't.
I know I'm probably weird for focusing so much on something technical like this, but it just bugs me so much.
Any help is greatly appreciated, really.
First of all, your replies are worth waiting for :) It's not the first time we interact, maybe you didn't notice but I sure did and I'm glad.
I should have specified, I understand that in MS rolling shutter is just hard to get, but not impossible. After all, the curtains don't sweep the sensor instantaneously. But I thank you very much for the detailed demonstration!
In general, I think I've figured it out now. The buckets are always open, but I had it backwards: we don't have lids to close to limit the exposure, we straight up flip the buckets and empty them to control how much light they contain. (Btw, I love the pseudo code you put there. My mind works a lot "by rules" and that code example is spot on, it's crystal clear to me. Thank you).
Let me try to explain so you can tell me if I understood or not:
In MS, the buckets are always open, in total darkness, until the shutter flies above them and lets a limited amount of light fall into them. The shutter is so fast, that the image each bucket sees is very chronologically close to the other ones. Once the shutter has passed, the buckets are back to total darkness, so no other light is collected, and the readout can piece together the light collected by every bucket. It does it slowly, but since no light is coming in anymore this does not cause any "distortion". Important point: the readout does not empty each bucket before reading it. It arrives at the next bucket it and measures the content.
In ES, the buckets are always open, in total light, then the readout starts and at each bucket, it empties it (it's probably full but since it's going to be emptied nobody cares nor sees what was collected) and then flips it back up for the specified amount of time before immediately reading it, then it goes on to the next bucket. This way, the light read by the readout is always the correct amount, but since it's collected at the very slow readout speed, the image that is pieced together will be chronologically skewed in the direction of the readout (top of the image is "before" the bottom in a tangible way). Important point: the readout does empty each bucket before reading it. It arrives at the next bucket, it empties it, and waits for the specified time interval before measuring the content.
So I guess the last piece I need is: there is a "way" to choose whether to empty the charge accumulated by each pixel when reading it or not. In MS it's chosen (at some mechanical level I suppose) that the readoud doesn't empty the charge accumulated before measuring it at each pixel, in ES it does empty it before measuring it. Not only that, in MS it's set that the readout happens immediately, while in ES it happens only after having waited the exposure time.
I'm way out of my element when it comes to microelectronics, circuitry, and engineering in this field (I don't even know what it's called), so probably even if I got a detailed answer to this I wouldn't be able to fully comprehend it.
If my intuition is correct, then I can say I finally have an answer to this (for me) burning question I've had for a while. I guess at the end there is indeed a fundamental difference in how the sensor works between MS and ES. I feel good thinking that at least my hunch was right, what I imagined was just so much more complicated that it felt impossible to be correct. Your explanation is much more "acceptable" for my brain :)
Apologies again for the delay, but I'm flattered to hear that you find the replies useful.
The reason why I keep harping on this is because ES vs MS are really very similar in how they operate. Empty, expose, measure. The only thing that's really different between the two is a MS can cover the photo sites after expose, which enables the shutter curtains to be ahead of the sensor's readout position.
Gotta empty the buckets ahead of time, but yup.
Yup!
The ES section is accurate as well, but...
and
Really truly: some ES shutters are just as fast, or nearly as fast, as a mechanical shutter. That's why some cameras are shipping without mechanical shutters now. Off the top of my head, some of the fastest reading sensors are: Z8/Z9 (1/250 and no shutter), A1 (1/240 and no mechanical front curtain), A9 i/ii (1/160), X-H2s (1/150), OM-1 (1/120), R3 (1/105).
I'm not sure if reading is used to empty the photo site, or if there's another mechanism to be honest. Either way, you need to empty, fill for the exposure duration, and read. If the second read happens to empty the site again it doesn't really matter that much.
The sensor is completely electrical, the big difference between ES and MS is the shutter mechanism. ES = use the sensor as the shutter. MS = use two mechanical curtains. In mechanical mode the photo sites can still be thought of as buckets. You gotta empty them before you expose them for the proper duration, otherwise you'll wind up with an improperly exposed image.
We're not talking anything beyond logical reasoning here. If you've been exposed to that mindset before, or just happen to think that way, you can probably wrap your head around this.
The big difference is that mechanical shutters can put a lid on the buckets, which enables the shutter to be ahead of the readout. The basic pattern is still the same: empty, expose, measure.
Just wait untli you start thinking about Electronic Front-Curtain mechanical shutters. In this case, the camera only uses the rear mechanical shutter curtain to block the senor - the front curtain uses the ES methodology.
Remember: empty, expose, fill.
Heh, sorry if I sound so pedantic, but I thought that would be my final message. Let me try with this one.
Yes, the steps are empty, expose, measure. I was just trying to explain the difference I'm finding in the electronic handling of these steps between MS and ES. In both cases though, the pixels experience the same three steps in the same sequence: empty, expose, measure.
The "only" difference I'm finding is that in MS the action of "measuring" the pixel happens effectively differently than in ES because, at each photo site, the action of "measuring" is only, well, reading sequentially the amount collected, which is limited in speed to the readout speed.
In ES, the action of "measuring" is preceded every time by "emptying, exposing"sequentially, all controlled electronically.
This makes me think that there is some liberty in choosing when to empty and expose pixels electronically, and it's not really limited neither by some kind of speed, nor some kind of sequence: in MS you don't really care because the curtains do the job, in ES you must control precisely when to empty a pixel and when to expose it.
This would also agree with how I understand EFC-S works: your closing of the curtains is limited by the curtain speed, so you have to empty and expose those pixels at the right time to let the curtain end the exposure so that you collect the specified amount of light. So if you're shooting at 1/1000 and the curtain closes at 1/250, you follow the ES method and empty+expose those buckets a tiny bit before the curtain passes over them. And since after the curtain it's darkness, just like in MS you can wait for the readout without worry.
Therefore, it is to my understanding that, electronically speaking, emptying and exposing the pixels can happen at very high speeds, independently from the measuring step: in MS, you empty and expose all of the pixels at once (from the sensor’s POV, of course it’s the curtain that does the exposure job), and then you measure them - the measuring is done once, "alone"; in ES you empty, expose, and measure each pixel individually - the measuring is coordinated with emptying and exposing (which also agrees with your beautiful pseudo-code).
EDIT: maybe it can be reduced to the emptying action alone. You can empty whenever you want, and since the pixel always gathers light as soon as you’re done emptying it, as long as you time your emptying action correctly the exposure can happen however you prefer: by the curtains in MS, by emptying at the right time so that the pixel is exposed correctly for when the readout happens in ES, and right before the curtain passes over in EFC-S
To me, in EFC-S this takes the best of both worlds: no fear of distortion because the reading phase is done in darkness, higher theoretical shutter (i.e. exposure) speed because you can empty+expose each pixel electronically even almost as the curtain closes above them.
I hope you understand that i truly appreciate your help, and I'm sorry I keep hammering at this stuff. I truly appreciate all the patience you can have for me.
I think you've got it, especially with the tweak in your edit. I'm happy to help where I can, hopefully my verbosity didn't get too in the way.
Your verbosity did not get in the way, at all. I appreciate every second you spent trying to help a random stranger on the internet understand something that, to you, was probably straightforward.
So thank you very very much and have a wonderful day!
We're all just a bunch of random people seeking to learn more, so I'm glad to hear it helped!