Industry News

August 29, 2017

Evolution of Zoom Camera in Smartphones

By Itay Yedid

From early days of the Samsung Galaxy K-zoom, through recent iPhone and Samsung Note8 dual cameras, to the future of folded zoom cameras: the technology evolution explained.

 Zoom is a commonly used photography feature; it allows the user to shift smoothly from a long shot to a close-up and vice versa. Optical zoom employs camera optics (a lens) to magnify the object, while digital zoom crops and upscales the input image. Unlike optical zoom, digital zoom reduces resolution and, as a result, diminishes image quality.

In recent years, mobile devices such as smartphones have become ubiquitous. Such devices typically include two compact cameras, a main rear-facing camera, and a secondary front-facing camera. Most compact cameras are designed with the traditional structure of a digital still camera — i.e., they comprise a lens placed on top of an image sensor. The lens focuses the incoming light to create an image of a scene on the sensor.

The dimensions of these cameras are largely determined by the size of the sensor and by the size of the lens. The size of the lens is mainly defined by the focal length (f) of the lens and the desired field of view (FoV). These properties are usually tied together such that the sensor size is proportional to FoV*f. A larger focal length results in a higher zoom factor and a longer lens, which necessitates a thicker camera.

As smartphone manufacturers are constantly striving to make their products thinner, a compact camera’s height becomes a limiting factor. Several approaches have been proposed to reduce compact camera height to alleviate this constraint. Nonetheless, due to the above limitation, traditional compact camera performance is very limited; the severe height restriction eliminates any distinct performance enhancement — in particular, an optical zoom feature.

Smartphone add-on zoom camera

Smartphone add-on zoom camera

This article covers the entire evolution of zoom cameras in smartphones, from early days of the Samsung Galaxy K-zoom, through the latest iPhones and the Samsung Note8, to the future of folded zoom cameras. It also explains some of the mobile photography fundamentals and how vendors strike a fine balance between them.

Traditional Smartphone Cameras
The common single compact digital camera’s structure includes a fixed focal length lens placed on top of an image sensor. This allows for easier optical assembly, reduces cost and increases the assembly yield.

Common compact digital camera's structure, fixed focal length lens on top of an image sensor

Common compact digital camera’s structure, fixed focal length lens on top of an image sensor

In spite of the clear advantages of this structure, it creates difficulties when trying to reach high zoom factors in a compact mobile camera. There are two main reasons for this:

  1. The lens has a fixed focal length and does not produce varying zoom factors.
  2. A variable focal length solution requires a significantly higher-than-usual camera “bump,” effectively increasing the overall smartphone thickness.

Camera module protrudes out of the smartphone encasement

Camera module protrudes out of the smartphone encasement

The lens in most mobile cameras can move slightly in relation to the image sensor position, enabling functions such as auto focus (AF) and optical image stabilization (OIS). A voice coil motor (VCM) is the dominant actuator technology for moving the lens in mobile cameras. The first generation of actuators allows AF only, moving the lens along the Z-axis (towards and away from the image sensor, or formally the image “sensor plane”). A more advanced generation of actuators can move the lens in 3 orthogonal axes: lens movement along the Z-axis for AF, and lens movement along the sensor plane’s axes (X, Y) to allow for OIS.

The AF function adds height to the camera module. To allow a range of images from an infinity shot of an object far from the camera (where the lens is closest to the image sensor) to a macro shot of an object located 10cm away from the camera (where the lens is furthest away from the image sensor), the lens must have a typical stroke of 250µm. This stroke is the aggregated sum of all heights.

OIS systems are incorporated to allow the user an equivalent optical configuration that is lower by 2 to 3 exposure values. By stabilizing the optical system, the user can increase the exposure time and allow for more light to reach the image sensor, while mitigating motion blur resulting from the handshakes. For typical optical stabilization, the lens moves around 100µm along the X and Y axes.

Camera module including auto focus (AF) and optical image stabilization (OIS) Moving the lens along the 2 OIS axes reduces blur due to minor shakes.

Camera module including auto focus (AF) and optical image stabilization (OIS)

Moving the lens along the 2 OIS axes reduces blur due to minor shakes

Prior Zoom Solutions
To date, typical smartphone cameras use a fixed focal length lens structure, where the lens is assembled on top of an image sensor having its optical axis positioned orthogonally to the sensor plane. This camera structure does not support changing the focal length and can only provide digital zoom.

Standard optical zoom cameras achieve different focal lengths (zoom) by allowing the movement of the lens elements along the optical axis. Changing the lens elements’ relative position requires a very accurate mechanism so that the optical performance is not compromised. Nevertheless, with this typical structure, a higher focal length results in a height penalty, or “camera bump,” in the smartphone.

In the past, there have been several attempts to implement such a solution in smartphones. For example, the Samsung Galaxy K-zoom announced in April 2014, which incorporated a typical pocket camera module, allowed 10x opto-mechanical zoom. This solution produced a product bigger than a pocket camera and was much thicker than a smartphone. The K-zoom had a sizable bump that more than doubled the smartphone’s thickness to 16.6mm — and much more when zooming – and increased its weight to 200 grams, compared to a typical 140-gram smartphone. The design resulted in a very thick camera with mediocre zoom performance given its large F/# (3.1-6.3), alongside the use of an image sensor with small pixels. This solution didn’t address most customers’ needs and failed to penetrate the market in high volumes.

Samsung Galaxy K Zoom allows 10 times optical zoom, resulting in a big bump and poor image quality

Samsung Galaxy K Zoom allows 10 times optical zoom, resulting in a big bump and poor image quality

Another attempt was made by the Asus Zenfone Zoom, announced in January 2015. It was the first phone with optical zoom that did not increase in height with the zoom operation. The camera, designed by Hoya, used folded optics to allow different zoom factors up to 3x zoom. At first glance, the technology looked promising, particularly since it offered optical zoom with OIS – an important feature for any smartphone user. But the device failed to impress, due to long zooming delays and the variable lens aperture (F/#2.7-F/#4.8), with poor low light performance and low resolution compared to other phones. These results prevented Zenfone Zoom from being a smartphone photography market leader.

Large image sensors measuring 1/1.2” (41Mpx) with a pixel size of 1.4µm marked another attempt to enable a zoom feature in smartphones. In February 2012, Nokia announced the 808 PureView, a smartphone with the biggest image sensor on the market. One of the aims in having such a large sensor was to allow digital zoom without degradation in resolution compared to a typical 1/3” (8Mpx) sensor. Cropping the center of an image from a 41Mpx camera allows an output of an 8Mpx image with FoV and resolution equivalent to 2x optical zoom. However, the sacrifice in thinness — the structure added a sizable bump, increasing the phone’s thickness to a whopping 17.95mm — and a high market price hampered the product’s success.

The next generation of Nokia 808 Pureview is the Nokia Lumia 1020, achieving zoom using a large image sensor.

The next generation of Nokia 808 Pureview is the Nokia Lumia 1020, achieving zoom using a large image sensor
Dual-Aperture Zoom Cameras
In 2014, Corephotonics introduced its dual-aperture zoom cameras, in which one camera has a wide FoV (a wide camera) and the other has a narrow FoV (a telephoto camera). This solution enables the user to switch between the wide and tele cameras to attain optical zoom, all the while maintaining a continuous zoom experience through image processing algorithms. The dual aperture configuration lends itself to additional benefits, such as depth perception and image fusion capabilities.

Corephotonics introduces its dual camera, February 2014

Corephotonics introduces its dual camera, February 2014

Starting in September 2016, various market leaders have adopted this zoom dual camera design concept, including Apple (iPhone 7 Plus), OPPO (R11), OnePlus (OnePlus 5), Xiaomi (Mi 6), Asus (ZenFone 4) and others. Dual aperture zoom camera technology is quickly becoming the mobile imaging solution preferred by smartphone OEMs.

Dual camera smartphones getting mainstream

Dual camera smartphones getting mainstream

Importantly, designing a zoom dual camera entails significant hardware challenges. Specifically, unless creatively designed, the tele camera creates a height barrier. The lens height, also known as the total track length (TTL), is defined as the distance between the object-side surface of the first lens element and the camera image sensor plane. In most miniature lenses, the TTL is larger than the lens’s effective focal length (EFL).

Typical lens design with TTL/EFL>1.0

Typical lens design with TTL/EFL>1.0

A typical TTL/EFL ratio for a given lens is around 1.3. In a single-aperture smartphone camera with a sensor measuring 1/3”, the EFL would typically be between 3.5mm and 4.0mm, leading to a FoV of 70-80°.


Assuming, for example, that one wishes to achieve 2x optical zoom using a dual-aperture smartphone camera, it would be natural to use a wide lens with an EFL of 3.5mm and a tele lens with an EFL of 7mm. However, without applying special techniques, the wide lens will have a TTL of approximately 4.55mm (3.5mm x 1.3), while the tele lens will have a TTL of approximately 9.1mm (7mm x 1.3). Integrating a 9.1mm lens into a smartphone camera would lead to a camera height of about 10mm, which has come to be unacceptable for any smartphone manufacturer. Therefore, there is a need for a tele lens assembly that can provide a TTL/EFL ratio of less than 1.0 and still achieve the same image quality of a standard lens in a fairly narrow FoV.

Miniature telephoto lens assembly Diagram from Corephotonics US Patent No. 9,402,032

Miniature telephoto lens assembly

Diagram from Corephotonics US Patent No. 9,402,032

Corephotonics’ solution integrates a tele lens with an EFL of 7mm into a camera module next to a standard wide lens, all in a Z-height measuring less than 6mm, which allows OEMs to integrate it into a smartphone without increasing the device’s thickness. Corephotonics was granted a patent in 2014 (US 9,402,032) for a TTL/EFL ratio below 1.0.

Telephoto lens design, TTL/EFL <1.0

Telephoto lens design, TTL/EFL <1.0

The image sensor is an important factor that should be carefully considered when designing a dual camera zoom configuration. In a dual camera configuration, the difference between the pixel size of the wide sensor and that of the tele sensor also plays a role in determining the zoom factor. Where the wide and tele sensors have the same pixel size, the zoom factor is determined based only on the EFL ratio. For example, if the wide camera has an EFL of 3.8mm and the tele camera has an EFL of 6.8mm, the zoom factor would be 6.8mm/3.8mm = 1.8 zoom factor. By comparison, using a wide sensor with a pixel size of 1.25µm and tele sensor with a pixel size of 1.12µm increases the zoom factor to 2x zoom: (6.8mm/3.8mm) x (1.25µm/1.12µm) = 2x zoom factor.

Dual-Aperture Zoom Camera Using a Folded Structure
Even when using a TTL/EFL ratio below 1.0, the resulting zoom factors would eventually be limited. In fact, given the average thickness of today’s smartphones, it is difficult to exceed 2x optical zoom (approximately 6.9mm tele EFL). Thus, to achieve higher zoom factors (3x-5x or EFL of 10-18mm), a different solution should be utilized.

Corephotonics’ patented folded camera solution (US 9,392,188) includes a wide camera and a folded tele camera. The folded optics tele camera structure includes a lens which has an optical axis perpendicular to the wide optical axis and an image sensor positioned along the tele optical axis (i.e, vertically). A 45-degree mirror or prism is used to “fold” the light coming from the object and direct it to the tele image sensor. This technique reduces the tele camera height, while increasing the zoom factor and maintaining uncompromising image quality.

Folded Tele camera (204) alongside a wide camera (202) Diagram from Corephotonics US Patent No. 9,392,188

Folded Tele camera (204) alongside a wide camera (202)

Diagram from Corephotonics US Patent No. 9,392,188

When placing the lens along a second optical path as in the tele camera, few challenges arise. In conventional compact camera modules, the diameter of each lens element is increased the closer it is to the image sensor. Therefore, when such a lens assembly is placed along a perpendicular optical path, the lens diameter, rather than the TTL, becomes a height-limiting factor. One possible solution is a lens element design with a diameter not exceeding the diameter of the aperture of the tele camera. This allows a dramatic reduction in the module height. In creating such a lens, it is important to maintain a low F# to increase the amount of light absorbed by the sensor, which has direct implications for overall image quality. Another point to consider is full coverage of the tele image sensor (for example, a 1/3” image sensor).

Tele lens design with lens elements diameters smaller than the aperture Diagram from Corephotonics US Patent No. 9,392,188

Tele lens design with lens elements diameters smaller than the aperture

Diagram from Corephotonics US Patent No. 9,392,188

In this new folded camera configuration, the actuation axes have been turned. For example, the AF axis, which traditionally contributes to the total module height and the smartphone’s thickness, is no longer a factor. On the other hand, if OIS is to be implemented, one of its axes becomes a determining factor with respect to the module’s height. Moreover, the OIS compensation stroke increases as the EFL increases. Typical OIS compensates for 1° camera pitch, yaw and rotation by shifting the lens along the X and Y axes. Therefore, it is easy to calculate that a camera with an EFL of 3.8mm will have about 66µm of OIS compensation in each axis . A camera with an EFL of 11.4m (3x optical zoom) will have approximately 200µm of OIS compensation, implying a height penalty of at least 400um.

To address the OIS compensation challenge, Corephotonics’ patented design utilizes the prism component. By rotating the prism around its axis, one can get the same OIS performance and even improved image quality.

OPPO's 5x folded zoom with 2 axes OIS and no camera module height penalty

OPPO’s 5x folded zoom with 2 axes OIS and no camera module height penalty

In a folded camera configuration, the image sensor requires further consideration. Unlike in a typical vertical camera, where the camera’s image sensor assembly area contributes to the module’s footprint, in a folded tele configuration, the image sensor’s assembly Y axis contributes to the handset’s thickness. There are methods to reduce the height penalty, such as rearranging the electrical interface and the electrical components surrounding it.

Using this folded camera technology, Corephotonics successfully achieved a 5x tele camera module including AF and OIS, all at a module height measuring less than 6.0mm.

 

Summary
The future of mobile dual cameras is brighter than ever. Adoption rates surpassed analysts’ predictions, and all major OEMs plan to launch or have already launched handsets with zoom dual cameras. Now the race begins for higher zoom factors, better optical stabilization, lower Z-heights and better low-light performance — anything that improves the overall camera and handset user experience. In this paper, we discussed some ways to increase the zoom factor and reduce the Z-height while maintaining superior image quality.

The road to improved camera hardware does not stop here. From reducing the overall footprint of the camera module, finding new ways to optically stabilize the cameras, giving the camera autonomous sensing capabilities and other innovative hardware improvements, Corephotonics will continue to provide state-of-the-art, never-before-seen camera module designs to help drive mobile imaging innovation forward.

— Itay Yedid is Mechanics Team Leader at Corephotonics

 Download The Evolution of Zoom Camera Technologies in Smartphones White Paper

Read More