Why is the iPhone’s "fourth camera" position given to lidar?

One, two, three, four... The number of flagship mobile phone cameras has been increasing since 2016 with about one camera per year. Some people ridiculed that Mahjong "eight tubes" will be the ultimate destination for all smartphone cameras.

Before the release of the new generation of iPhone this year, many people speculated that the iPhone 12 Pro Max will be equipped with a fourth camera. Unexpectedly, the iPhone 12 Pro series placed a LiDAR (LiDAR) in the precious "fourth camera" position.

Playing cards without "common sense" has always been Apple's "common sense."

What is LiDAR?

Viewed vertically, the circular black area at the bottom right of the iPhone 12 Pro series mobile phone camera module is LiDAR           

LiDAR stands for Light Detection And Ranging, LIDAR Chinese name, is emitting the laser beam characteristic amount of the target position detection, speed radar system, a laser transmitter, an optical receiver, the turntable and the information processing system, etc., may Obtain the target's distance, azimuth, height, speed, posture, and even shape parameters for detection, tracking and recognition.

Regardless of technical terms, LiDAR on the iPhone 12 is widely understood. It is a tiny device that can detect the distance of an object and record depth information, about the size of a mobile phone camera.

When working, LiDAR can fulfill two important functions.

1. Measure the approximate distance of the object from the phone

Measure the distance by emitting a laser beam invisible to the naked eye

2. Record the depth information of things within the projection range

At present, the LiDAR range given by Apple is the farthest five meters. In other words, the iPhone 12 Pro series mobile phones equipped with LiDAR can theoretically measure and record distance and depth information within a range of about 5 meters.

What is the difference between LiDAR and ToF?

When it comes to estimating object distance and recording depth information, many Android manufacturers have equipped ToF (Time-of-Flight) sensors on some models to achieve these two functions.

For example, the P40 Pro has a 3D depth-sensing camera (ToF), which can perceive the depth information of objects in depth and achieve a large-scale depth of field effect.

At present, most mobile phone manufacturers use iToF (indirect time-of-flight sensor), while LiDAR belongs to dTof (direct time-of-flight sensor).

Although both iToF and LiDAR are ToF, the difference is not small.

First, the "signals" sent out are different.

iToF sends out a light signal modulated by a sine wave, and the intensity of light and dark changes regularly. LiDAR emits a laser beam.

dToF principle

Second, the principle of estimating distance is different.

iToF "estimates" the distance from emission to reflection by sensing the difference of sine wave light signals. LiDAR is the direct measurement of distance by laser.

iToF principle

Third, the accuracy range, measurement rate and anti-interference ability are different.

The signal of iToF is easy to be interfered. The accuracy of the measurement decreases as the distance becomes longer, and the measurement is relatively slow. It is more suitable for distance measurement in the centimeter range.

The laser beam of LiDAR has strong anti-interference ability. Theoretically, the measurement accuracy does not decrease with the distance. The measurement is faster, and it is more suitable for long-distance ranging.

Finally, iToF is easier to miniaturize, and dToF is more difficult to integrate and miniaturize, so there are obvious commercial differences between the two. iToF is mostly used in mobile phones, and dToF is more commonly used in large devices such as cars and drones.

Car radar

LiDAR is faster, more accurate and farther than iToF, but it is more difficult to integrate and more expensive. Apple was able to integrate LiDAR into the iPhone 12 camera module, it must have been a lot of thought.

Why did Apple choose LiDAR?

Perhaps the iPhone will become a four-five-six-camera phone one day, but at a critical juncture when people are discussing "Is the  iPhone behind taking pictures? ", choosing to give the precious "fourth camera" position to LiDAR is enough to show Apple's future layout How important.

What does Apple want to do?

On the official webpage of the iPad Pro 2020, Apple gave the answer straightforwardly. LiDAR currently serves AR (Augmented Reality).

At the moment when mobile phone photography almost reached the ceiling, Apple chose a different path from other manufacturers. Instead of choosing high-resolution and multiple optical zoom, it added LiDAR to allow users to record "depth information."

The benefit of users being able to record in-depth information is very obvious, I sum it up as "two-way integration".

One is reality virtualization, that is, AR, which can record various information about real things with a camera system and LiDAR, and then display it on a virtual screen to observe close-up from any angle, which also breaks the limitations of time and space.

Without the in-depth information of iPhone 12, it is impossible to use AR to watch virtual iPhone

The other is virtual reality, that is, VR, which can add real-dimensional information to virtual things, allowing virtual things to "live" the characteristics of real things on the Internet.

With depth information, the photo is no longer a plane

Dynamic emoji can be seen as a simplified version of VR, virtual characters have facial depth information data of real characters

Both of these fusions are inseparable from in-depth information, which is also a bridge between AR and VR. For example, a teacher uses an AR device that can record in-depth information to take selfies and live teaching. Students wear VR glasses to listen to a lesson. Every student sees the teacher as an avatar, but the avatar’s facial expressions and action details are the same, which is difficult Simply use AR or VR to describe briefly.

Let's talk about mobile phones

What is a mobile phone for human society?

This question is difficult to answer. When it is small, it is just a convenient communication tool. When it is large, its impact on human society can be said to be extremely far-reaching.

Looking back at human history after the advent of mobile phones, the common demand of users for information processing and exchange has always been decentralization.

At the beginning, the common demand of users was to improve the efficiency of information exchange, so the core function of mobile phones was to make calls and send text messages. The central node of information exchange-fixed-line telephones, mails, faxes, post and telecommunications tied up by wires are completely marginalized, and they are now just a supplementary tool in a special scenario.

Next, the common needs of users are to break the imbalance of information and further decentralization. Everyone wants to have their own voice. Therefore, the core function of mobile phones has become social, and a large number of social apps have emerged.

It is precisely because Apple's full-touch screen product iPhone, at the right time, accurately and efficiently solves the user's common needs and pain points in the second stage, Apple can beat Nokia and become the market's first comprehensive product.

Now, what will be the common needs of users?

Users need to further decentralize, not only to speak, but also to record and pass on their individual civilization as much as possible, so that their individual civilization can leave a record in the history of human civilization.

iPhone commercials show iPhone’s life recording function

A long time ago, all living beings had no sense of history, let alone the tools to record themselves. The advent of mobile phones has turned the data stored on the server into "personal history books". These small "personal history books", which are as small as ants, exert unparalleled influence and together form a human "great history book"-human society In the common large database, individuals are no longer "nameless" that can be directly ignored in history, nor are they such-and-such that can be written off with a swipe of a pen, but are living and living in the common "great history book" of mankind.

Now, LiDAR has given us a higher-dimensional "writing" ability, adding depth information to dimensions such as sound, plane images, and behavior habits, breaking through the limitations of planes.

Immortality, starting from in-depth information

For ordinary users, the current intuitive help brought by LiDAR is not very obvious, and more of it belongs to the degree of "being better, and not having much impact."

The potential benefits it brings are very obvious. If the conditions and camera app allow, every scene, every moment, every person encountered, every flower seen, every touch A piece of cutlery and every selfie can be recorded by LiDAR. As long as the time and technology are mature, the amount of data is large enough, coupled with data such as sound, image, behavior and habits, a virtual world can be created in the server. It is exactly the same as the virtual self, the real world and the real self, even waiting for the day when the level of technology is high enough, you can create yourself in the real world.

If the robots in "Western World" have a corresponding real human, then getting the in-depth information of the real human is the first step in making a robot

If technology is more advanced one day, you can give this virtual self a set of "humanity algorithms." A hundred years later, someone thinks of us, and they only need to enter the virtual world to see us who are "alive". Why not do it? Another form of immortality?