We tested different scenarios and the finding is quite surprising! It’s all about the implementation of the computation. Just going crazy with the algorithms manipulating the image don’t help at all.
What is computational photography?
Computational photography involves the use of specialized software algorithms to enhance the camera performance. The main goal is to bypass the limitations of smaller optical equipments with the help of software processing.
And the use of computational techniques has taken smartphone photography to a whole new level. A lot of features in the smartphone camera wouldn’t even exist without the use of specific machine learning algorithms. Smartphone manufacturers are relying on these computational techniques more and more to improve the camera performance year on year.
However, these image processing techniques becomes useful only with their proper implementation.
Problems with the over-use of computation:
Do not over do it!
Often, we see images that look over-processed and unnatural. It isn’t necessary to mention that this worsens the overall experience.
Sharpening becomes over-sharpening
Smartphone sensors usually have very small photosites, resulting in a failure to capture the required amount of information. As a result, the images lack detail compared to dedicated cameras with the same resolution.
To compensate for this, manufacturers tune the camera software to apply some level of sharpening to the image. However, when this sharpening exceeds a certain limit, it causes clearly visible artifacts. Just look at the image above for reference. The branches look almost unearthly! Over-sharpening has resulted in white boundaries on both sides of the image that do not even exist.
The image above has relatively simple structures. The more fine objects there are, the messier it looks. However, by using a different camera application like a Google Camera port that uses different image processing algorithms, you can achieve improved results.
Balancing colors and structure
Sometimes the structure gets lost on the subject in order to boost it’s colors. And the boosted colors don’t look pretty either.
The two images represents two different approaches to computational photography. The first image prioritizes boosting colors while trading the structures on the petals for it. The second one keeps the color accurate while also maintaining the structures much better.
Though the sensor and lenses are the same, yet the final result looks vastly different, right? All because of the two different approaches. I repeat, Just going crazy with the algorithms manipulating the image don’t help at all.
It’s not about what change you can do, rather it’s all about what change improves the image. It depends on the understanding of what is to be done and by how much.
Using the algorithms too far without understanding the needs may overall worsen the image quality. This is the case for most of the low to mid-tiar devices where manufacturers put more effort in associating the abbreviation ‘AI’ with the camera than tuning the algorithms.
However, you can use the controls you have in pro mode to create an image that suits your taste better.
Over-processing is overloading the device:
The extra software processing techniques puts extra load on the processor, it consumes more power to execute the extra calculations. Using processing heavy algorithms causes poor performance of the camera application and sometimes it even affects the final result.
These are noticeable on devices where the processing load exceeds the raw computing power of the device.
The result of overloading the device
Shutter lag
We have seen a lot of devices suffering from shutter lag while taking more than one image back to back. It happens because of the workload surpassing the processing capability of the device. If you tap the shutter and move the device away from the subject without giving it the time to complete processing, it may leave you with an image with motion blur.
Having even a bit noisy one would also be better than having a completely unusable one.
Shorter battery life:
The more calculations it has to perform, the more power it’s going to consume. For a device that runs on battery, the power consumption is definitely an important thing. The extra power consumption while using the camera results in a shorter battery life.
Thermal throttling:
No electronic equipment is cent per cent efficient meaning that there will always be a certain percentage of power making it’s way to heat generation. Now, the more power is used, the proportion that converts into heat becomes also higher. It results in a higher temperature gain of the device when it’s using more power for a given task.
Final verdict:
Devices are becoming more and more powerful in terms of their processing capability. So taking advantage of that to make up for optical limitations is great for a smartphone.
But often times the final result looks overprocessed and unnatural due to an inappropriate use of the computational processing methods. Knowing what change needs to be done is important. Over-use of these software processing methods also puts extensive loads on the SoC that worsens the overall experience.
Computational photography is great, but only with the proper implementation
