Blending Intel Real Sense 3D sensor with Magic Mirror in video using OpenCV 4.

In this tutorial, I show how to blend the Intel Depth Camera D435i and the Elgato Cam Link 4K + Mirascreen WIFI Display Receiver video input source in Opencv 4.

Prior to attempting this tutorial, I assume you have the following hardware at hand:
Elgato Cam Link 4K
Depth Camera D435 or D435i
Mirascreen G5 Plus

Cam Link 4k
D435 Intel Realsense 3D stereo sensor
Mirascreen wifi dongle

Step 1: Ensure OpenCV and Python 3 are running on your Windows machine.

Tutorials:
https://www.learnopencv.com/install-opencv-4-on-windows/

Why Windows?
I tested the Cam Link 4K in linux and the image was not great. I have not done this on a Mac.

Step 2: Install the Intel® RealSense™ SDK with Python 3 wrappers.

If everything went well in step 1, we are now ready to install support for the Intel 3D stereo sensor.

SDK link:
https://github.com/IntelRealSense/librealsense

pyrealsense2 python module:
https://pypi.org/project/pyrealsense2/

Tutorials:
This tutorial below uses python 2, but following the same steps, one can install the library for python 3.
https://jessicayin.com/realsense-opencv-installation

Prior to proceeding, please verify that you can run the example code under:
https://github.com/IntelRealSense/librealsense/tree/master/wrappers/python/examples

Step 3: Connecting the Cam Link

Elgato tutorial:

Elgato Cam Link video tutorial

Local Cam Link + Wifi Dongle computer connection:

Cam Link + Mirascreen wifi dongle

Opencv video source test:

Setting up video in opencv tutorial:
https://www.pyimagesearch.com/2016/02/22/writing-to-video-with-opencv/

Cam link + Wifi Dongle as opencv source:

Ensure you can see cam link in opencv
Connect to the wireless dongle
Wait for connection to establish

Running opencv again after connection should show the content of your wireless display. In our case, it will show the third desktop connected via wifi.

Step 4: Setup MagicMirror in windows.

Complete walk through:
https://forum.magicmirror.builders/topic/4089/complete-walkthrough-install-magicmirror-on-a-pc-windows-7-10

Magic Mirror configuration modules:

Wallberry Theme
Now Playing on Spotify

Step 5: Merging D435 input source and Cam Link.

OpenCV image blending tutorial:
https://www.learnopencv.com/alpha-blending-using-opencv-cpp-python/

Transparent overlay in OpenCV tutorial:
https://www.pyimagesearch.com/2016/03/07/transparent-overlays-with-opencv/

Step 6: Blending the output of the align-depth2color.py example for the librealsense sensor and the display output of the wifi dongle.

Code for aligning depth to color:
https://github.com/IntelRealSense/librealsense/blob/master/wrappers/python/examples/align-depth2color.py

Github to reproduce blending in 1280X720:
https://github.com/huguensjean/algin_depth2color.git

Special thanks:

Adrian Rosebrock
Andrew Kan
Jessica Yin
Satya Mallick