# 2D Optical Calibration

{% hint style="info" %}
These Modules require python 3.7 in order to work
{% endhint %}

{% hint style="info" %}
The realsense T265 & T261 can only be used with on application at a time, you'll have to close realsense viewer. 
{% endhint %}

{% embed url="<https://www.youtube.com/watch?v=0JzzaQhaRtU>" %}

Above: A video walking you through the 2D calibration process for northstar.

## Setup

1. Download the **realsense-integration** branch of the following repo: \
   <https://github.com/BryanChrisBrown/ProjectNorthStar/tree/realsense-integration>
2. Setup python with the following dependencies: \
   `1) Pyrealsense2`\
   `2) opencv-contrib-python`\
   `3) numpy`
3. Print the calibration stand and [intel realsense t265 mount](https://drive.google.com/file/d/1emnKXaevUyY_jEExWeCeesK_AUDbetcu/view?usp=sharing) **m3x12mm** screws to mount the t265 itself.
4. If using a Deck X and t261 mount, use the [intel realsense t261 mount](https://drive.google.com/file/d/1BSTmZ2WVgoRmrjVorQG1AWA08u4qBiEE/view?usp=sharing) instead, the screws are included in your Deck X Kit.&#x20;

## Calibrating your headset

Run through the following steps to calibrate your headset.&#x20;

* **captureGraycodes.py -** run **`python captureGraycodes.py`** to run this script
* Ensure that your headset is placed on the calibration stand, with the stand's camera looking through it where the users' eyes will be.<br>
* On line 44 there is a line of code that changes the window offset for the graycode generator.  \
  &#x20;This example here moves the window 1920 pixels to the right, and 0 up and down. You'll want the X value (1920) to be your main monitor's width in pixels. `cv2.moveWindow ("Graycode Viewport", 1920, 0)` <br>
* It helps to place a piece of cloth over the rig to shield the cameras + headset from ambient light.<br>
* The sequence of binary codes will culminate in a 0-1 UV mapping, saved to **"./WidthCalibration.png" "./HeightCalibration.png"** in your main folder.<br>
* **calibrateGraycodes.py**
* Running this script will fit a 3rd-Degree 2D Polynomial to the left and right "eye"'s X and Y distortions.<br>
* This polynomial will map from each display's 0-1 UV Coordinates to rectilinear coordinates (where a 3D ray direction is just (x, y, 1.0)).

When you are finished, you may paste the output of the calibrateGraycodes.py into [this diagnostic shadertoy](https://www.shadertoy.com/view/wsscD4) to check for alignment. Additionally, ensure that your headset is plugged in and displaying imagery from your desktop. Running this script will display a sequence of gray codes on your North Star, capturing them at the same time. &#x20;

![The shader toy should look like this when viewed through the headset](https://1971497060-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-Lm0OvojFPoDKE9jIY4P%2F-MBo4fRPon_LZqhyzIST%2F-MBo4ptmC7lJ3bORYN_L%2Funknown-2.png?alt=media\&token=4a9227cf-edba-4514-8208-15f218d9dfe2)

Additionally, there should be a `NorthStarCalibration.json` in this directory which you may use in the Unity implementation.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.projectnorthstar.org/project-north-star/calibration/2d-calibration.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.