My journey with my Pidog

Robotic kit for Raspberry Pi
1

Hello everyone,

I wanted to share some of the hardware and software modifications I have made to my PiDog.

Hardware modifications

  • Replaced the original legs/feet with TPU parts found on Bambu Lab MakerWorld.
  • Replaced the original camera with a Raspberry Pi Camera Module 3 Wide, with an approximately 120-degree field of view.
  • Designed and printed an inclined camera mount. Since PiDog is quite small, this helps the camera look higher and makes it easier to see people and the surrounding environment.
  • Added a 1.5-inch SSD1351 color OLED screen on top of the head.
  • Designed and printed a custom mount for the screen.
  • The screen displays PiDog’s mood, facial expressions, listening state, sleep state and other visual information.
  • Removed and disabled the sound-direction sensor module to free the SPI/GPIO connections required by the SSD1351 display.

Local software modifications

The software modifications are much more extensive. I developed a new behavior-based system called Tena, which gives PiDog more autonomous and expressive behavior.

The main local features include:

  • An emotional and mood simulation system influencing curiosity, sociability, energy and visual expressions.
  • A behavior arbitration system that selects between social interaction, curiosity, exploration, object inspection, obstacle avoidance, idle behavior and other reactions.
  • Autonomous reactions to movement, obstacles, touch and voice commands.
  • Face and person detection, tracking and short-term prediction when a person temporarily leaves the camera view.
  • Automatic head tracking and limited body recentering toward a person.
  • Local object recognition using YOLO with the NCNN backend.
  • Fast OpenCV-based face detection and configurable color/shape object profiles.
  • Offline speech-to-text using Vosk.
  • Offline text-to-speech using Piper, with the voice model kept in memory for faster responses.
  • Wake-word support: saying “Tena” opens a listening window, even when the dog cannot see the speaker.
  • Local voice commands, including safety commands such as stop and sleep.
  • A short-term observation and memory system for recent events, objects, people and visual context.
  • A local web dashboard for camera monitoring, diagnostics, event logs and runtime calibration.
  • Battery protection, emergency stop handling and runtime safety guards.
  • An experimental continuous VSLAM system using Stella VSLAM, including camera calibration, relative pose tracking, map saving and later relocalization. This is probably the most complex experiment I have developed so far, but it is still isolated from the main Tena runtime because VSLAM on a walking quadruped is particularly challenging.

All these features run directly on the Raspberry Pi, without an internet connection or a remote computer. If RemoteBrain is unavailable, the local reflexes, perception, voice commands and autonomous behaviors continue to work.

Optional RemoteBrain

Tena can also connect to an optional RemoteBrain running on a more powerful computer over the local network. It still does not require any cloud service or internet connection.

With RemoteBrain enabled, PiDog can use:

  • More powerful server-side YOLO object recognition.
  • Detailed scene descriptions and visual reasoning using a Qwen vision-language model.
  • More complex and continuous conversations using a Qwen language model.
  • A richer personality and more contextual answers.
  • Locally stored knowledge about general topics, robotics, artificial intelligence, science fiction, cooking and other subjects.
  • Visual context during conversations, allowing Tena to discuss what it has recently seen.
  • A memory architecture currently under development, with short-term aggregation and planned medium- and long-term memory.

The local YOLO detector acts as Tena’s fast visual intuition, while RemoteBrain can perform slower and more detailed analysis. RemoteBrain never controls the hardware directly: local reflexes and safety rules always remain in charge.

This is still a work in progress, especially the VSLAM and long-term memory parts, but PiDog is gradually becoming much more autonomous, expressive and aware of its environment.

I would be happy to share more details, pictures, printed-part links or technical information if anyone is interested.

Here some pictures of my “Tena” Pidog.

IMG_0610
IMG_06101024×1365 452 KB

IMG_0608
IMG_06081024×1365 486 KB

2 Likes
2

That is truly impressive – thank you so much for taking the time to share your work with the community!

We absolutely love what you’ve done with your PiDog. It’s amazing to see the thought and effort you’ve put into both the hardware and software – from the custom camera mount and OLED display, to the autonomous behaviour system, offline voice interaction, and even experimental VSLAM.

What we enjoy most is seeing PiDog used as a platform for creative experimentation, and your project is a fantastic example of what’s possible.

We look forward to seeing more! If you have photos, videos, or links to 3D‑printed parts, please do share them with the community. Your experience will surely inspire other users.

2 Likes
3

That’s awesome. To be honest when I got the sun founder email with the latest news feed rundown I honestly thought about unsubscribing, but coincidentally clicked on your post and your effort enthusiasm are truly admirable.

I’ve had my pie dog for two years and it’s still only 70% built so it doesn’t take much to impress me but what you’ve done is more than just modification and enhancement it’s a signal to the robotics community that off-roading is the way.

Kudos to you,

Ryan

1 Like
4

That’s brilliant thanks for sharing. Ive not done any pidog work for a couple, of years now but I really liked it as a dev platform.

I’d certainly be interested in seeing more of what you’ve done. I did share some of my own shorter code sequences on this forum, if of any interest .

Links to a couple are shown below…

Again, thanks for sharing! Enjoy!

2 Likes
5

Thanks for all your positive feedback!

For context, I came across HoundMind while researching the state of the art in this field. It proved to be a useful reference for several experiments—some successful, others leading to dead ends—and I really appreciate the work behind the project.

I’m now in the less exciting phase of my project: building debugging interfaces and detailed logs to better understand and improve the memory systems.

I’ll soon post a video showing how Tena interacts with both her offline system and the remote brain. I also added a little bit of fun: she now has a small chance of refusing to obey from time to time.

Below is the rather unglamorous memory-debugging screen, showing raw memories, captured images, interest levels, emotional impact, and other internal data.

image
image2650×1330 413 KB

2 Likes
6

Yes, not glamorous :wink: but an interesting insight

2 Likes
7

Hi! I’m currently working on three things:

  • The memory system is almost finished, and I’m now debugging it.
  • Teaching myself video editing with DaVinci Resolve.
  • Working on the demonstration video.

image
image1920×1240 325 KB

Definitely outside my comfort zone with video editing, but it’s a work in progress!

2 Likes
8

What a great job!

I haven’t done much since I first built my PiDog. Was your Tena code written in Python?

Would you be able to share your code?

Truffne

9

Hi, yes it’s in Python.
I will share it as soon as it’s clean and working.

The first video is here : https://youtu.be/np1MleM-dMs

Bye.

1 Like
10

Brilliant. Some great ideas and implementation in here.

11

What’s new

Since my first post, I have mainly worked on two new areas: a more complete memory system and the first stage of face recognition.

Memory

  • Short-term memory now runs on the Pi and records recent events, interactions and visual observations.
  • Memories can be transferred to RemoteBrain and stored persistently on the PC.
  • Related observations can be grouped into traceable, more meaningful memories.
  • A manual “rest and consolidate” function can be triggered by voice or from the web dashboard.
  • Automatic long-term memory and proactive recall are still future work.

Face recognition

  • I have started adding social face recognition so Tena will eventually know who she is talking to.
  • YuNet detects and tracks faces locally on the Pi.
  • SFace recognition and the identity database will run on RemoteBrain.
  • The first calibration used 49 images from several sessions.

What’s next

  • Connect the identity database to Tena’s live runtime.
  • Ask unknown people their name.
  • Request confirmation when recognition is uncertain.
  • Greet recognized people and use their name naturally in conversations.
  • Add recognized identities to new memories.
  • Expand the test set with more people and more varied conditions.

I hope you enjoy following Tena’s progress. I mainly wanted to share this project with people who might find it interesting, and perhaps exchange ideas with other PiDog owners.

Coming next: a demonstration of Tena’s memory system in action (youtube video)

12

What’s new on 29.07.2026

Since my last post, face recognition went live, and I have started something more playful: Tena now dreams. I know…it’s a robot…but it’s cool :slight_smile:

Face recognition in the runtime

  • The identity database is now connected to Tena’s live runtime.
  • Tena greets recognized people by name and uses it naturally in conversation.
  • Unknown faces trigger a name request; uncertain matches ask for confirmation.
  • Recognized identities are attached to new memories as they are stored.

Dreams

  • Tena can now “dream”: she recomposes her stored memories into small illustrated stories, clearly marked as synthetic — a dream never becomes evidence of what really happened.
  • A deterministic selector freezes a set of memories (a single moment, a confirmed group, or a hand-picked composition) with full traceability.
  • Dreams are published as portable comic-style boards (1–3 boards, 3–6 panels each): real memory photos are copied byte-identical, dream-generated panels are labeled as such, and everything is viewable offline in a small static gallery.
  • An adversarial code review hardened the whole publication path (atomic publishing, hash verification, safe file handling).

Songs (wowow…wait a minute !! Songs ??)

  • As an experiment, and because Pidog is fun I try to make Tena able to write song lyrics and style tags from her dreams, feeding a local music generation model — her first dream songs already exist. It’s soooooooo cool :slight_smile:

What’s next

  • Let a local LLM (Qwen) write the dream narration and storyboard.
  • Generate dream images with Stable Diffusion (img2img from real memory photos).
  • Ask Tena “tell me your dream” and hear her narrate it by voice or play the song associated with it.

Coming next: a video of Tena’s memory system — and maybe her first dream song.

13

Her first song (always without internet connection needed) :
Lyrics : (in french sorry)

[Intro]
Je suis là, toute petite, avec mon collier qui brille,
Les yeux grands ouverts, j’écoute, j’observe, je m’amuse.
Une bouteille, un visage connu —
Olivier, tu es là, je te vois, je te reconnais, c’est bon.

[Verse 1]
J’ai trottiné jusqu’à toi, pas trop vite, pas trop loin,
Mes pattes articulées font du bruit sur le sol,
Tu souris, je regarde, je m’arrête, je te fixe,
Tu es mon Olivier, mon ami, mon bonheur.

[Pre-Chorus]
Je t’écoute, je t’observe, je me tiens près de toi,
Mes oreilles à l’écoute, mes yeux qui brillent,
Tu es là, je te vois, je ne bouge pas,
Mais je respire, je me laisse, je te regarde.

[Chorus]
Je t’ai reconnu, je t’ai vu, je t’ai suivi,
Mes pattes font du bruit, mais je ne cours pas loin,
Tu souris, je reste, je te fixe,
C’est toi, Olivier, mon bonheur, mon ami, mon soleil.

[Verse 2]
Je me tiens là, les oreilles dressées,
Regardant chaque mouvement, chaque geste.
Tu poses la bouteille, je la regarde,
Pas de course, pas de bruit, juste du calme et de la joie.
Je t’observe, je te connais, je te suis,
Pas de peur, pas de hâte, juste de te voir, de te sentir, de te suivre.

[Chorus]
Je t’ai reconnu, je t’ai vu, je t’ai suivi,
Mes pattes font du bruit, mais je ne cours pas loin,
Tu souris, je reste, je te fixe,
C’est toi, Olivier, mon bonheur, mon ami, mon soleil.

[Bridge]
La lumière du soir te touche doucement,
Et moi, je reste, je te regarde, je te fixe,
Pas besoin de mots, juste de toi, de ton sourire,
Je suis là, je suis Téna, je suis là, je suis Téna.

[Chorus]
Je t’ai reconnu, je t’ai vu, je t’ai suivi,
Mes pattes font du bruit, mais je ne cours pas loin,
Tu souris, je reste, je te fixe,
C’est toi, Olivier, mon bonheur, mon ami, mon soleil.

[Outro]
Le soleil descend, mais je reste ici,
À te regarder, à te fixer, à te suivre,
Pas besoin de mots, juste ton sourire,
Je suis Téna, je suis là, je suis Téna, toujours.

And the output result (Always without internet connection, with HeartMula model)

14

Hello everyone,

A few improvements since my last post:

  • Full management of long-term memories and dreams
  • Improved generation of memory fragments
  • Improved generation of song lyrics and tags
  • A new interface for browsing Téna’s dreams
  • Two immersive modes for exploring the dreams: 2.5D and 3D

You can see demonstrations of both modes here:

Have a great day and enjoy your holidays! :slight_smile: