# 2D Optical Calibration

{% hint style="info" %}
These Modules require python 3.7 in order to work
{% endhint %}

{% hint style="info" %}
The realsense T265 & T261 can only be used with on application at a time, you'll have to close realsense viewer. 
{% endhint %}

{% embed url="<https://www.youtube.com/watch?v=0JzzaQhaRtU>" %}

Above: A video walking you through the 2D calibration process for northstar.

## Setup

1. Download the **realsense-integration** branch of the following repo: \
   <https://github.com/BryanChrisBrown/ProjectNorthStar/tree/realsense-integration>
2. Setup python with the following dependencies: \
   `1) Pyrealsense2`\
   `2) opencv-contrib-python`\
   `3) numpy`
3. Print the calibration stand and [intel realsense t265 mount](https://drive.google.com/file/d/1emnKXaevUyY_jEExWeCeesK_AUDbetcu/view?usp=sharing) **m3x12mm** screws to mount the t265 itself.
4. If using a Deck X and t261 mount, use the [intel realsense t261 mount](https://drive.google.com/file/d/1BSTmZ2WVgoRmrjVorQG1AWA08u4qBiEE/view?usp=sharing) instead, the screws are included in your Deck X Kit.&#x20;

## Calibrating your headset

Run through the following steps to calibrate your headset.&#x20;

* **captureGraycodes.py -** run **`python captureGraycodes.py`** to run this script
* Ensure that your headset is placed on the calibration stand, with the stand's camera looking through it where the users' eyes will be.<br>
* On line 44 there is a line of code that changes the window offset for the graycode generator.  \
  &#x20;This example here moves the window 1920 pixels to the right, and 0 up and down. You'll want the X value (1920) to be your main monitor's width in pixels. `cv2.moveWindow ("Graycode Viewport", 1920, 0)` <br>
* It helps to place a piece of cloth over the rig to shield the cameras + headset from ambient light.<br>
* The sequence of binary codes will culminate in a 0-1 UV mapping, saved to **"./WidthCalibration.png" "./HeightCalibration.png"** in your main folder.<br>
* **calibrateGraycodes.py**
* Running this script will fit a 3rd-Degree 2D Polynomial to the left and right "eye"'s X and Y distortions.<br>
* This polynomial will map from each display's 0-1 UV Coordinates to rectilinear coordinates (where a 3D ray direction is just (x, y, 1.0)).

When you are finished, you may paste the output of the calibrateGraycodes.py into [this diagnostic shadertoy](https://www.shadertoy.com/view/wsscD4) to check for alignment. Additionally, ensure that your headset is plugged in and displaying imagery from your desktop. Running this script will display a sequence of gray codes on your North Star, capturing them at the same time. &#x20;

![The shader toy should look like this when viewed through the headset](https://1971497060-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-Lm0OvojFPoDKE9jIY4P%2F-MBo4fRPon_LZqhyzIST%2F-MBo4ptmC7lJ3bORYN_L%2Funknown-2.png?alt=media\&token=4a9227cf-edba-4514-8208-15f218d9dfe2)

Additionally, there should be a `NorthStarCalibration.json` in this directory which you may use in the Unity implementation.