Once upon a time, a rock skid, made a spark, and humankind learned to create fire. This is the fifth installment of our ARKit series. Today, we will walk you through how to implement light estimation in augmented reality with ARKit.
Light estimation enhances your graphics’ blending with the real world in AR — with shading algorithms utilization. When your app renders graphics, you can use the rendering information and shading algorithms to match your camera’s captured real-world lighting conditions with your scene graphics.
I hope you’ll enjoy this ARKit tutorial. And hopefully, this ARKit tutorial can also spark up an amazing idea like that rock.
Now let’s begin.
Prerequisites
This ARKit tutorial builds on previous ARKit tutorials knowledge. If you happen to get stuck anywhere, feel free to check out the ARKit tutorial series to help you out.
What You’ll Implement and Learn
In building out this tutorial’s ARKit light estimation project, we will do the following:
- Place a sphere node on top of a detected horizontal plane.
- Illuminate the sphere node with a light node
- Test out the intensity and temperature light properties
- Update and implement UIs
- Finally, implement light estimation inside a SceneKit’s scene rendering method.
Getting Started
First, download the starter project here. I have already built the UI of the app and created the action method of the buttons.
Build and run. You should be prompted to allow camera access in the App. Tap OK to allow camera access in your App.
Creating a Sphere Node
First, we will start by creating a sphere in augmented reality. Open up the ViewController.swift file in Xcode. Add the following method inside the ViewController class:
func getSphereNode(withPosition position: SCNVector3) -> SCNNode {
let sphere = SCNSphere(radius: 0.1)
let sphereNode = SCNNode(geometry: sphere)
sphereNode.position = position
sphereNode.position.y += Float(sphere.radius) + 1
return sphereNode
}
The getSphereNode(withPosition:) method does the following:
- Take in a position parameter.
- Create a sphere geometry with a 0.1 CGFloat radius.
- Create a sphere node with the sphere geometry we created earlier.
- Set the sphere node’s position to the position argument value.
- Add the sphere’s radius value onto the sphere node’s y position value so that the sphere is right on top of detected horizontal surfaces.
- Increase the sphere node’s y position by one. This way, the sphere node will to sit one meter above detected horizontal surfaces.
- Return the sphere node.
In brief, this method create a sphere and place it on top of a detected horizontal plane.
Adding Light Node
Next, we are going to add a light source (i.e. SCNLight) to illuminate the scene. Create the following method inside the ViewController class:
func getLightNode() -> SCNNode {
let light = SCNLight()
light.type = .omni
light.intensity = 0
light.temperature = 0
let lightNode = SCNNode()
lightNode.light = light
lightNode.position = SCNVector3(0,1,0)
return lightNode
}
The way to illuminate a scene is by attaching lights to SCNNode objects using their light property. This is what this method is about. Let me explain what the getLightNode() method does in details:
- First, we create a SceneKit light object (i.e.
SCNLight) with its type set to omni. An omni light type illuminates a scene from a point in all direction. There are other light types including directional, spot and ambient. - Next, we set the light object’s intensity and temperature property value to zero.
- In order to use the light object to illuminate the scene, we create a light node object and set the light source to the
lightproperty of the node. - We also set the light node object’s y position to one meter above its parent node.
Now, let’s add another method inside the ViewController class:
func addLightNodeTo(_ node: SCNNode) {
let lightNode = getLightNode()
node.addChildNode(lightNode)
lightNodes.append(lightNode)
}
This method calls the getLightNode() method to get a light node and add the light node to the given node. We also append the light node to the array of light nodes.
Next, add the following inside the renderer(_:didAdd:for:) method right below the planeAnchorCenter constant:
let sphereNode = getSphereNode(withPosition: planeAnchorCenter) addLightNodeTo(sphereNode) node.addChildNode(sphereNode) detectedHorizontalPlane = true
The code above does the following:
- Get a sphere node with the plane anchor center position
- Add a light node onto the sphere node
- Set the mapped anchor node as the sphere node’s parent node
- Set detected horizontal plane variable to true
Testing Light Properties
Now, let’s test out the effects ambient intensity and color temperature has on rendered graphics. Before that, update the ambientIntensitySliderValueDidChange(_:) method like this:
@IBAction func ambientIntensitySliderValueDidChange(_ sender: UISlider) {
DispatchQueue.main.async {
let ambientIntensity = sender.value
self.ambientIntensityLabel.text = "Ambient Intensity: \(ambientIntensity)"
guard !self.lightEstimationSwitch.isOn else { return }
for lightNode in self.lightNodes {
guard let light = lightNode.light else { continue }
light.intensity = CGFloat(ambientIntensity)
}
}
}
The code above runs on the main thread and altogether sets the light nodes’ light intensity property value to the slider’s sender value. Also, update the ambientColorTemperatureSliderValueDidChange(_:) method like this:
@IBAction func ambientColorTemperatureSliderValueDidChange(_ sender: UISlider) {
DispatchQueue.main.async {
let ambientColorTemperature = self.ambientColorTemperatureSlider.value
self.ambientColorTemperatureLabel.text = "Ambient Color Temperature: \(ambientColorTemperature)"
guard !self.lightEstimationSwitch.isOn else { return }
for lightNode in self.lightNodes {
guard let light = lightNode.light else { continue }
light.temperature = CGFloat(ambientColorTemperature)
}
}
}
The codes above runs on the main thread and altogether sets the light nodes’ light temperature property value to the slider’s sender value.
Cool! Let’s build and run the project. Point the device’s camera to a horizontal surface. Upon a horizontal plane detection, you should be able to see a floating sphere. Feel free to play with the sliders to get a feel for the light’s intensity and color temperature properties.
Showing/Hiding the Light Estimation Switch
For now, the ambient intensity and color temperature controls are always displayed. But for the light estimation switch, it is hidden by default. What I want to do is to display the controls when a horizontal plane is detected. Therefore, update the detectedHorizontalPlane property’s didSet method like this:
var detectedHorizontalPlane = false {
didSet {
DispatchQueue.main.async {
self.mainStackView.isHidden = !self.detectedHorizontalPlane
self.instructionLabel.isHidden = self.detectedHorizontalPlane
self.lightEstimationStackView.isHidden = !self.detectedHorizontalPlane
}
}
}
This light estimation stack view contains a UISwitch object as well as a UILabel object. We set the light estimation stack view to show when detectedHorizontalPlane has been set to true.
Working with the Light Estimation Switch
Now we are going to implement the light estimation switch. Add the following inside the lightEstimationSwitchValueDidChange(_:) method:
ambientIntensitySliderValueDidChange(ambientIntensitySlider) ambientColorTemperatureSliderValueDidChange(ambientColorTemperatureSlider)
On light estimation switch value change, we update the light nodes’ light intensity and temperature properties value to their respective slider value.
Implementing Light Estimation
Okay, what’s left is the light estimation implementation. First things first, why light estimation? As I mentioned at the very beginning of this tutorial, light estimation enhances your graphics’ blending with the real world in AR. You want to make those graphics match the real-world lighting conditions. For instance, if you dim the lights of your room, you want to reflect the light condition on the virtual object to make it more realistic.
You can get the estimated scene lighting information from the captured video frame. Now add the following method inside the ViewController class:
func updateLightNodesLightEstimation() {
DispatchQueue.main.async {
guard self.lightEstimationSwitch.isOn,
let lightEstimate = self.sceneView.session.currentFrame?.lightEstimate
else { return }
let ambientIntensity = lightEstimate.ambientIntensity
let ambientColorTemperature = lightEstimate.ambientColorTemperature
for lightNode in self.lightNodes {
guard let light = lightNode.light else { continue }
light.intensity = ambientIntensity
light.temperature = ambientColorTemperature
}
}
}
The updateLightNodesLightEstimation() method runs on the main thread and does the following:
- Make sure the light estimation switch’s state is on.
- Safely unwrap the current scene view sessions frame’s light estimate.
- Extract the unwrapped light estimate’s ambient intensity and ambient color
temperature property values. - Loop through the light nodes.
- Safely unwrap the light node’s light property.
- Set the light’s intensity property to the ambient intensity constant.
- Set the light’s temperature property to the ambient color temperature constant.
Next, call the following method inside the renderer(_:updateAtTime:) method:
updateLightNodesLightEstimation()
The renderer(_:updateAtTime:) method gets called exactly once per frame in SceneKit before any animation, action evaluation, or physics simulation. Light estimation is to be constantly applied to our scene. Hence,the updateLightNodesLightEstimation() method is called inside of the renderer(_:updateAtTime:) method.
As with all UI updates, it is best practice to do the UI updates on the main thread. We call the updateLightNodesLightEstimation() method inside the asynchronous method.
Trying out the App Demo
That’s it! Now it’s time to try out the complete demo. Build and run the project. Upon a horizontal plane detection, you can switch the light estimation switch on and see light estimation in effect.
You can try the light estimation by switching your lights on/off.
Final Words
I hope you have enjoyed and learned something valuable from my tutorial. Feel free to share this tutorial on your social networks so that your circle can make some knowledge gains too!
For the complete Xcode project, you can download it on GitHub. If you have any questions or feedback, please leave me a comment below.
