If you’re a LEGO MINDSTORMS enthusiast, then the SerialTalk Micropython library is your ticket to taking your custom hardware projects to the next level. Developed to serve as a universal integration tool, SerialTalk offers unparalleled versatility and ease of use. Let’s dig in!
One Library to Rule Them All
SerialTalk was born out of the need for a single, cohesive way to bring all our custom hardware projects under one roof. Whether you’re working with UART, Sockets, Bluetooth Classic, BLE, or any other serial protocol, SerialTalk has you covered. When it comes to MINDSTORMS Robot Inventor, this library is the fastest and most versatile method for brick-to-brick communication over Bluetooth.
Making It Work
Here’s the best part: You can simply define functions and their return formats, then call them from another Python program. Sounds simple? It is!
Here’s a small code snippet to demonstrate SerialTalk with LMS-ESP32 and MINDSTORMS EV3:
# On the LMS-ESP32 side
from serialtalk.auto import SerialTalk
st = SerialTalk()
def get_temperature():
return 22.5 # Replace this with your sensor data
# Pass the function and the struct type of the result. Here it's f for floating point number.
st.add_command(get_temperature, 'f')
st.loop()
# On the MINDSTORMS EV3 side, using Pybricks on ev3dev.
from serialtalk.auto import SerialTalk
from pybricks.parameters import Port
st = SerialTalk(Port.S1)
ack, temp = st.call('get_temperature')
print(f"Temperature is {temp}")Application of SerialTalk: Smart Camera ball grabber
With the Smart Camera Ball grabber, I’m using SerialTalk to pass the size of the ball and the location on the camera sensor to the MINDSTORMS EV3 brick. Communication is so fast that I can easily implement a feedback loop on the EV3 and steer towards the ball smoothly.
The OpenMV camera also has an I2C voice recognition sensor plugged in. SerialTalk also transfers the voice command it recognizes to the EV3 brick. Now the brick is like a pet! I can say: “Hey brick, go forward!” Or “Hey brick, fetch ball!”
You can see how fast the EV3 reacts to the camera and voice commands on Instagram.
SerialTalk: A Micropython Toolbox for Your Projects
SerialTalk is super handy for a wide array of applications. We use it mostly to talk to our LMS-ESP32 board or our OpenMV camera. See the full SerialTalk library on GitHub.
- Sensors
- Displays
- Smart Cameras
- Computer Vision
- Remote Control
- Custom Electronics
Some of our older projects use UartRemote for the same purpose. SerialTalk is 100% compatible with UartRemote. SerialTalk is just more versatile and better maintained.
Cool SerialTalk Micropyton Project Ideas
Let’s get those wheels turning! Here are some project ideas you could try with SerialTalk:
- Weather Station: Use sensors to collect weather data and display it on your MINDSTORMS robot.
- Security Bot: Integrate a smart camera for facial recognition.
- Remote-Controlled Car: Control your LEGO car from your computer or phone.
Got a cool project? Share your creations on social media and tag @antonsmindstorms. I’m eager to see and share your innovative designs!
Get Involved with Serial Talk
Interested in diving deeper? You can find all the documentation you need on docs.antonsmindstorms.com. Whether you’re a beginner or a seasoned pro, there’s something for everyone to learn and contribute.
So there you have it! SerialTalk Micropython library offers a streamlined, efficient, and highly versatile way to level up your MINDSTORMS projects. Go unleash your creativity!
Very excited to give SerialTalk a try. What if you want to communicate with both a LMS ESP32 and a OpenMV from your Mindstorms Hub? I’m guessing you need to use a separate port on the hub for each one. Can you use Serial Talk for both boards or do you need to use UartRemote for one board and Serial talk for the other. (In the past, I tried to use uartRemote for both boards and it didn’t work.).
I suggest this:
– Use a separate port for each one.
– Create a serial interface for each port using
from projects.mpy_robot_tools.serialtalk.mshub import MSHubSerial– And then
st_port_f = SerialTalk(MSHubSerial('F'))andst_port_e = SerialTalk(MSHubSerial('E'))– See also: https://docs.antonsmindstorms.com/en/latest/Software/SerialTalk/docs/index.html
Do you have any plans to post step by step tutorials for SerialTalk projects like you did for UartRemote? The tutorials were a huge help and I probably couldn’t have started using UartRemote without them. Also, can I just use the version of Python that comes with LEGO Mindstorms Inventor instead of PyBricks?
Does this help? https://docs.antonsmindstorms.com/en/latest/Software/SerialTalk/docs/index.html
You can use LEGO python, but only SPIKE2 and MINDSTORMS. SPIKE3 is nerfed.
What are you using to physically interface the EV3 to the LMS-ESP32 in your image at the top? I purchased a triple pack of your LMS-ESP32s last year, but only recently am working with them again on my RIS Hub. But I recently picked up a couple of EV3 Hubs and they want to join in on the fun 🙂
We’re using a special adapter that is not (yet) for sale in the shop. Do you want to buy one? I could put it up, but I was unsure about how many people would want one, and if I wouldn’t be doing al the effort for nothing. If you program the EV3 in regular python (not pybricks-micropython) you can also use the usb-A port on the side of the EV3 to connect to the LMS-ESP32. Linux will create a serial interface for you.
I am not a hard core programmer, so am always on the lookout for hardware, tutorials and functioning examples that make my programming life funner and easier when interfacing with otherwise proprietary and often incompatible stuff (looking at you LEGO and your range of Mindstorms ;P )
So far I prefer to use PyBricks for all my LEGO stuff, and just assumed that this adapter was needed for this Serial Talk to work on and EV3. My EV3 (running ev3dev) has not been used much as my focus has been on interfacing the ESP32 and other electronics with LEGO RIS Hub, so thought this Serial Talk might help to bring the EV3 into that fold.
And while I can’t guarantee I would get one right away (but probably soon), I guess it depends on knowing more about what it contributes and costs (and of course I would have to import over the pond).
But I can guarantee that no one will ever purchase one if it is never listed 😉
I am trying to connect EV3 hub to the Openmv HT plus camera. I am assuming I need to add serialtalk.py to the openMV code. When I try adding the line:
from serialtalk.auto import SerialTalk
I get the importerror: no module named ‘serialtalk’ message. The program and the serialtalk.py files are in the same directory. They are also both samed to the openmv camera. If I REM the statement the program runs and I see an image. OpenMV sample program below. Once I get the camera working and the EV3 can follow a ball I was hoping to add the voice command module as soon above. It would be helpful to get a full tutorial on this specifically for the EV3
EV3_Openmv_camera – By: Wareman – Sun May 26 2024
import sensor, image, time
#from serialtalk.auto import SerialTalk
#st = SerialTalk
sensor.reset()
sensor.set_pixformat(sensor.RGB565)
sensor.set_framesize(sensor.QVGA)
sensor.skip_frames(time = 2000)
clock = time.clock()
while(True):
clock.tick()
img = sensor.snapshot()
print(clock.fps())
You need to copy the whole serialtalk folder over to the lms-esp32. https://github.com/antonvh/SerialTalk/tree/master/serialtalk
Yes, but I do not have the EV3 to LMS-ESP32 adapter. The above image shows the Openmv connected to the EV3 using the Openmv EV3 board and the voice control module also connected to the Openmv EV3 board. How does the EV3 receive information from either the Openmv or the voice command module. Also if I use the LMS-EPS32 how does the Openmv camera connect to the LMS-ESP32 board, and then connect to the EV3 hub?
I will include the adapter in your next order. You can also solder a wire – I’ll make a post on that. And finally, it should be possible to use USB on EV3, but I haven’t got around to doing that. If you use the adaptor or a soldered wire, you can use pybricks-micropython and VS Code. The camera or the voice command module will be accessible as an UARTDevice. https://docs.pybricks.com/en/v3.0/iodevices/uartdevice.html
I have copied all the files from the serialtalk directory to the LMS-ESP32. I still cannot get the
from serialtalk.auto import Serialtalk on either the LMS-ESP32 or on the EV3 hub. Both sides complain that it cannot find the serialtalk.auto module. What if I remove the .auto from the import? Are there parameters that need to be added? If so, what should I use?
You can look inside the auto.py file and find the statements and files you really need. When you copy Serialtalk to a device, you need to copy the folder with the init.py inisde. The folder name needs be ‘serialtalk’ lowercase. This is how python imports modules and packages: https://docs.python.org/3/tutorial/modules.html#packages
Okay, I now have the serialtalk folder and contents saved to the LMS-ESP32. When I run the test program above I do not get an error or return to the system prompt in Thonny. Using VS Code I accidentally copied the whole serialtalk folder — docs, examples, images… I then deleted everthing except the serialtalk folder. VS Code is saying that I have 28 changes (Source Control) is this only for my local copy? Afraid to make changes to your github files! As for the EV3 I have the same setup as on the LMS-ESP32. First off I needed to add this line before the code would even attempt to download and run:
#!/usr/bin/env pybricks-micropython
Then I get a invalid syntax error for this line:
print(f”Temperature is {temp}”)
If I remove the f I just get this as ouput on the VS Code terminal:
Temperature is {temp}
As best as I can tell the code is running on the LMS-ESP32. What am I missing?
Google for Python format strings. You can also write: “Temperature is {}”.format(temp) This is more compatible.
Don’t worry about chaning github files. It is your local copy and you don’t have push rights. You don’t need to delete any files if you only copy the serialtalk folder that is a subfolder of the SerialTalk folder.
I still cannot get the simple code shown above to work. All I would like to see if that data can move from the EV3 hub to and from the LMS-ESP32. I still get an invalid syntax error with the suggested print line change. If I remove the formatting and just use print(temp) I get a delimiter not found error. The LMS-ESP32 is running the above code, and the EV3 cable adapter is plugged into the LMS-ESP32 as seen in the very top image.
I don’t see any delimiters. Maybe share the complete code through facebook, patreon or preferrably mattermost.