squid-game-doll

An attempt to create a "Red Light, Green Light" robot inspired by Squid Game TV series, using AI for player recognition and tracking.

Geometry

• Expected play area 10 x 10 m indoor
• In order to hit a 50 cm wide target @ 10m the laser shall be precise 2.8° in horizontal axis. This should be doable with standard servos and 3D-printed pan&tilt platform for the laser (see hardware folder).

Detecting the red laser dot

In order to reliably point the laser to the eliminated player, laser dot position must be acquired, positioning error calculated, and angles corrected accordingly. In the example picture below, red laser dot is found on the webcam and a visor is added on top of predicted position.

Tools used

• Python 3.11, libraries Numpy, OpenCV-python. See laser-pict-gen.py

pip install opencv-python numpy

• Logitech webcam HD PRO Webcam C920 on Windows 11

• Green laser 5mW (11 EUR) : https://aliexpress.com/item/1005005346537253.html . This model has high luminiosity with respect to red laser, but has poor focus. This may be better for eye safety.

• Red laser 5mW (3 EUR) : https://aliexpress.com/item/1005008087745092.html . This model has good focus.

• ESP32C2 MINI Wemos board for servo control.

Current approach

• Choose a channel from the webcam picture (R,G,B) or convert to grayscale.

• Apply a threshold to the picture to find the brightest pixels

diff_thr = cv2.threshold(channel, threshold, 255, cv2.THRESH_TOZERO)

Resulting image:

• Increase remaining spots with dilate (a key ingredient!!)

masked_channel = cv2.dilate(masked_channel, None, iterations=4)

Resulting image:

• Look for circles using Hough Transform

circles = cv2.HoughCircles(masked_channel, cv2.HOUGH_GRADIENT, 1, minDist=50,
                                param1=50,param2=2,minRadius=3,maxRadius=10)

param2 is a very sensitive parameter. minRadius and maxRadius are dependent on webcam resolution and dilate step.

• If the circles found are more than one, increase threshold (dichotomy search) and retry.
• If no circles are found, decrease threshold (dichotomy search) and retry
• If threshold limits are reached, exit reporting no laser
• If exactly one circle is found, exit reporting the circle center coordinates

What is tricky about laser recognition

• Exposure of the webcam is very important:

If picture is too bright (autosettings tend to produce very bright images), laser detection fails. For this reason, webcam is set to manual exposure and underexposed. Probably, an exposure calibration step is required until the picture has the right average brightness. With Logitech C920 results are OK in interior room with exposure around (-10, -5) @ 920x720 resolution. This will vary with different webcam.

Overexposure (-4)	Under-exposure (-11)
(threshold)	(after dilate)

• Exposure is somehow dependent on resolution requested and FPS requested to the webcam. I fixed the parameters in the webcam initialization step to avoid variability.

• Some surfaces absorb more light than other resulting in brightest spot not being the laser. Additional search algorithms to be tested (e.g. checking maximum R to G / R to B color ratios)

• Green laser seem to work better than Red laser on many surfaces. But it may be my Aliexpress green laser is more powerful.

• Try-and-error loop is slow - another approach which helped me speed up testing is to generate pictures by adding fake laser spots (ellipsis with variable red/brightness) and compare actual position with predicted precisions.

Dev notes regarding laser detection

• Very slow startup on x64 / Windows 11 fixed by

import os
os.environ["OPENCV_VIDEOIO_MSMF_ENABLE_HW_TRANSFORMS"] = "0"

import cv2

• Image processing techniques that did not work

Attempt	Why it failed	What could be done
Switching on the laser programmatically and substract images to find the spot	Even without buffer, webcam images have latency > 250 ms resulting in difference images having lots of pixels especially with persons in the scene	Solve webcam latency and retry with fast laser switch (>25Hz?). Check if the laser really turns off immediately.
Laplace transform to find rapid variations around the spot	It's more for contour detection and it finds a lot of rapid variations in normal interior scenes, or faces	???
HSV thresholds based on fixed value	Red laser is not fully red on the picture, white is present at the center	Implement adaptive adaptation on V value?