I wanted to share some of the hardware and software modifications I have made to my PiDog.
Hardware modifications
Replaced the original legs/feet with TPU parts found on Bambu Lab MakerWorld.
Replaced the original camera with a Raspberry Pi Camera Module 3 Wide, with an approximately 120-degree field of view.
Designed and printed an inclined camera mount. Since PiDog is quite small, this helps the camera look higher and makes it easier to see people and the surrounding environment.
Added a 1.5-inch SSD1351 color OLED screen on top of the head.
Designed and printed a custom mount for the screen.
The screen displays PiDog’s mood, facial expressions, listening state, sleep state and other visual information.
Removed and disabled the sound-direction sensor module to free the SPI/GPIO connections required by the SSD1351 display.
Local software modifications
The software modifications are much more extensive. I developed a new behavior-based system called Tena, which gives PiDog more autonomous and expressive behavior.
The main local features include:
An emotional and mood simulation system influencing curiosity, sociability, energy and visual expressions.
A behavior arbitration system that selects between social interaction, curiosity, exploration, object inspection, obstacle avoidance, idle behavior and other reactions.
Autonomous reactions to movement, obstacles, touch and voice commands.
Face and person detection, tracking and short-term prediction when a person temporarily leaves the camera view.
Automatic head tracking and limited body recentering toward a person.
Local object recognition using YOLO with the NCNN backend.
Fast OpenCV-based face detection and configurable color/shape object profiles.
Offline speech-to-text using Vosk.
Offline text-to-speech using Piper, with the voice model kept in memory for faster responses.
Wake-word support: saying “Tena” opens a listening window, even when the dog cannot see the speaker.
Local voice commands, including safety commands such as stop and sleep.
A short-term observation and memory system for recent events, objects, people and visual context.
A local web dashboard for camera monitoring, diagnostics, event logs and runtime calibration.
Battery protection, emergency stop handling and runtime safety guards.
An experimental continuous VSLAM system using Stella VSLAM, including camera calibration, relative pose tracking, map saving and later relocalization. This is probably the most complex experiment I have developed so far, but it is still isolated from the main Tena runtime because VSLAM on a walking quadruped is particularly challenging.
All these features run directly on the Raspberry Pi, without an internet connection or a remote computer. If RemoteBrain is unavailable, the local reflexes, perception, voice commands and autonomous behaviors continue to work.
Optional RemoteBrain
Tena can also connect to an optional RemoteBrain running on a more powerful computer over the local network. It still does not require any cloud service or internet connection.
With RemoteBrain enabled, PiDog can use:
More powerful server-side YOLO object recognition.
Detailed scene descriptions and visual reasoning using a Qwen vision-language model.
More complex and continuous conversations using a Qwen language model.
A richer personality and more contextual answers.
Locally stored knowledge about general topics, robotics, artificial intelligence, science fiction, cooking and other subjects.
Visual context during conversations, allowing Tena to discuss what it has recently seen.
A memory architecture currently under development, with short-term aggregation and planned medium- and long-term memory.
The local YOLO detector acts as Tena’s fast visual intuition, while RemoteBrain can perform slower and more detailed analysis. RemoteBrain never controls the hardware directly: local reflexes and safety rules always remain in charge.
This is still a work in progress, especially the VSLAM and long-term memory parts, but PiDog is gradually becoming much more autonomous, expressive and aware of its environment.
I would be happy to share more details, pictures, printed-part links or technical information if anyone is interested.
That is truly impressive – thank you so much for taking the time to share your work with the community!
We absolutely love what you’ve done with your PiDog. It’s amazing to see the thought and effort you’ve put into both the hardware and software – from the custom camera mount and OLED display, to the autonomous behaviour system, offline voice interaction, and even experimental VSLAM.
What we enjoy most is seeing PiDog used as a platform for creative experimentation, and your project is a fantastic example of what’s possible.
We look forward to seeing more! If you have photos, videos, or links to 3D‑printed parts, please do share them with the community. Your experience will surely inspire other users.
That’s awesome. To be honest when I got the sun founder email with the latest news feed rundown I honestly thought about unsubscribing, but coincidentally clicked on your post and your effort enthusiasm are truly admirable.
I’ve had my pie dog for two years and it’s still only 70% built so it doesn’t take much to impress me but what you’ve done is more than just modification and enhancement it’s a signal to the robotics community that off-roading is the way.
For context, I came across HoundMind while researching the state of the art in this field. It proved to be a useful reference for several experiments—some successful, others leading to dead ends—and I really appreciate the work behind the project.
I’m now in the less exciting phase of my project: building debugging interfaces and detailed logs to better understand and improve the memory systems.
I’ll soon post a video showing how Tena interacts with both her offline system and the remote brain. I also added a little bit of fun: she now has a small chance of refusing to obey from time to time.
Below is the rather unglamorous memory-debugging screen, showing raw memories, captured images, interest levels, emotional impact, and other internal data.
Since my first post, I have mainly worked on two new areas: a more complete memory system and the first stage of face recognition.
Memory
Short-term memory now runs on the Pi and records recent events, interactions and visual observations.
Memories can be transferred to RemoteBrain and stored persistently on the PC.
Related observations can be grouped into traceable, more meaningful memories.
A manual “rest and consolidate” function can be triggered by voice or from the web dashboard.
Automatic long-term memory and proactive recall are still future work.
Face recognition
I have started adding social face recognition so Tena will eventually know who she is talking to.
YuNet detects and tracks faces locally on the Pi.
SFace recognition and the identity database will run on RemoteBrain.
The first calibration used 49 images from several sessions.
What’s next
Connect the identity database to Tena’s live runtime.
Ask unknown people their name.
Request confirmation when recognition is uncertain.
Greet recognized people and use their name naturally in conversations.
Add recognized identities to new memories.
Expand the test set with more people and more varied conditions.
I hope you enjoy following Tena’s progress. I mainly wanted to share this project with people who might find it interesting, and perhaps exchange ideas with other PiDog owners.
Coming next: a demonstration of Tena’s memory system in action (youtube video)
but an interesting insight