How to use the Huskylens with LEGO SPIKE Prime or Robot Inventor

Anton

Updated on:

Huskylens LEGO SPike prime how to

After my first experiments with the Huskylens, here’s a complete guide to connecting the smart AI camera to your LEGO robot. For this guide, I’m using the new hub. The camera also works with EV3, but I haven’t built that yet. Read on to find out what hardware you need, how to do pin-out and wiring, and how to program the hub with Python.

Required hardware to do image recognition on your LEGO hub

How to make wire for the Huskylens by yourself

Below is the pin-out that I used for connecting the Huskylens to SPIKE Prime and Robot Inventor. There is one option without dc-dc converter and one with the converter. The Huskylens needs 3v3 in the power supply and the RX and tx wires connected. The camera is 5v resistant, but I found that communication hampered. I didn’t have a fixed 3v3 regulator, so I used a larger but tunable converter.

I did not use the included JST-wire but a spare one that I had lying around. That’s why colors are different. I want to use the included wire to connect to EV3 in a later project.

How to wire the Huskylens using the Distance Sensor Breakout board

If you don’t want to cut and solder wires, you can also use the Distance Sensor Breakout Board. It now comes with a free screwdriver to open up the distance sensor. You can simply slide the four Huskylens wires over the header pins, in the order they are. For a more stable performance of the Huskylens, you can easily wire in a 3V3 buck converter. If you set the port to 100% PWM, M+ will be about 8V and M- will be 0V (GND).

How to program your Robot Inventor or SPIKE hub to work with the Huskylens

LEGO does not provide Scratch word blocks for third-party hardware, nor do the producers of the Huskylens. As far as I know, it is not even possible to import plug-ins into the LEGO Software with your own blocks.

The good news is that LEGO officially supports Python now. This code only works on LEGO MINDSTORMS Robot Inventor app and firmware! You can flash your SPIKE hub with that firmware if necessary. To make things easy, I have created a LEGO-Huskylens library on Github.

To use that library, simply copy it in front of your script, or use the installer script to make the library importable.

Once the script is installed you can use the boilerplate script to start coding your smart camera with LEGO. Here’s an example script that follows a line.

# default imports
from mindstorms import MSHub, Motor, MotorPair, ColorSensor, DistanceSensor, App
from mindstorms.control import wait_for_seconds, wait_until, Timer
from mindstorms.operator import greater_than, greater_than_or_equal_to, less_than, less_than_or_equal_to, equal_to, not_equal_to
import math

# imports from Anton's Mindstorms LEGO Huskylens library
# after installing through https://github.com/antonvh/mpy-robot-tools/blob/master/Installer/install_mpy_robot_tools.py
from projects.mpy_robot_tools.pyhuskylens import (HuskyLens,
ALGORITHM_FACE_RECOGNITION,
ALGORITHM_FACE_RECOGNITION,
ALGORITHM_OBJECT_TRACKING,
ALGORITHM_OBJECT_RECOGNITION,
ALGORITHM_LINE_TRACKING,
ALGORITHM_COLOR_RECOGNITION,
ALGORITHM_TAG_RECOGNITION,
ALGORITHM_OBJECT_CLASSIFICATION,
ALGORITHM_QR_CODE_RECOGNITION,
ALGORITHM_BARCODE_RECOGNITION,
ARROWS, # key for get() dict
BLOCKS, # key for get() dict
FRAME, # key for get() dict
clamp_int)

### huskylens usage ###
# Init with the the port it is in. Here it is in port 'E': hl = HuskyLens('E')
# Add a pwm parameter if you use a wire with a voltage converter: hl = HuskyLens('E', pwm=100)
# get_blocks(ID=None, learned=False) returns a list of block objects
# blocks have x, y , width , height, ID properties.
# get_arrows(ID=None, learned=False) returns a list of arrow objects
# arrows have x_tail, y_tail , x_head , y_head, ID and direction properties

# get(ID=None, learned=False) returns a dict with blocks, arrows & frame

# on firmware 0.5+ you also have these Huskylens class methods available:
# show_text("text", position=(10,10))
# clear_text()

# Create your objects here.
ms_hub = MSHub()
hl = HuskyLens('E', pwm=100)
ma = Motor('A')
mb = Motor('B')

# Write your program here.
ms_hub.speaker.beep()

# Get direction of a LEARNED arrow. Point the lens at a line and press the learn button.
print("Starting line tracking")
hl.set_alg(ALGORITHM_LINE_TRACKING)
while not ms_hub.left_button.is_pressed():
arrows = hl.get_arrows(learned=True)
if len(arrows) > 0:
direction = ((arrows[0].x_head+arrows[0].x_tail)/2 - 160)/-6
ma.start_at_power(clamp_int(30+direction))
mb.start_at_power(clamp_int(-30+direction))
else:
ma.start_at_power(0)
mb.start_at_power(0)
ma.start_at_power(0)
mb.start_at_power(0)

You can see the line follower script in action on my Youtube channel. Here’s a video. Did you subscribe?

What to do if you get stuck trying to connect Huskylens to SPIKE Prime or Robot Inventor

If it doesn’t work for you, leave a reaction in the comments below. You can also support me on Patreon for an ultra-fast reply. If you make something cool or find out new things about this camera, be sure to tag me on Facebook, Instagram, or YouTube. I’d love to see what you are doing with this.

Like this article? Help us make more!

Your support matters

We appreciate your support on Patreon and YouTube! Small things count.

Become a Patron

Don't miss a thing

Subscribe below to get an email when we publish a new article.

Share this with someone who needs to know

13 thoughts on “How to use the Huskylens with LEGO SPIKE Prime or Robot Inventor”

  1. Hi Anton,
    Your work here is awesome. Really impressive.

    Now that LEGO Education have released the SPIKE App v3 with a different implementation of MicroPython, are you likely to update this project to work with that version?

    Or the more evergreen PyBricks?

    I’ve tried to access features such as .pwm and .uart and setting up duplex ports using SPIKE v3 but LEGO don’t seem to have included libraries or functionality for this. Do you know if they are there? Are those things accessible at a lower level?

    BTW, I am having a few problems with your sample. I get the following errors:

    Expected OK, but got: b” No header
    Huskylens connection failed. Check wires and port is E?
    Start line tracking

    The only change I’ve made to your code is line 36 is now hl = HuskyLens(‘E’, 9600, False, 100)
    It is in the correct port because it powers the Huskylens and there is communication because the code changes the camera to line tracking.

    Any ideas?

    Reply
  2. Hi Anton,

    Correction, your line following code above works correctly. I used the LF code on your github and had not trained the Huskylens camera on a line correctly.

    Reply
  3. We intend to create a facial recognition project using the Huskylens AI vision sensor and Spike Prime. However, we have encountered an issue with the firmware and LEGO Education Spike software of the Spike Prime controller, as they have been updated to version 3.0 and no longer support the “Pybricks” library (which was integrated into the version 2.0 firmware). This means that Spike Prime cannot drive the external vision sensor, Huskylens. Have you come across this problem? Is there a plan to update and upgrade the Pybricks library to establish compatibility between Spike Prime version 3.0 and Huskylens? Alternatively, are there any other solutions available?

    Reply
  4. For someone that is very new to lego robotics: could you please elaborate the the esp32 board is related to the SPIKE Smart Camera Breakout Board? Are both components required to connect the hub?

    Reply
    • You only need one. The LMS-ESP32 is more comfortable and has more usage options, since you can emulate LEGO sensors and connect bluetooth and wifi. It is a bit more expensive, however.

      Reply
  5. Anton hi
    when I run the installer I get Import Error: no module named ‘ubinascii’
    Can you please tell me what I am doing wrong
    thank you

    Reply
  6. What I am not understanding is the following;
    1. where do I copy the files the HuskyLens folder I downloaded (the one with the library)
    2. Which MicroPython environment do I use. (pybricks, Spike 3.0 python …)

    I apologize but as a beginner it’s very confusing.
    Thank you

    Reply
  7. May I use the same steps with the Distance Sensor Breakout Board when trying to connect an OpenMV Camera to the Spike Prime hub? If not, may you create a new article about it? Thank you!

    Reply

Leave a Reply

Item added to cart.
0 items - 0.00