New project announcement: k9-polyvox

News
1

Meet k9-polyvox: Give Your PiDog a Personality (or Twelve!)

Hello fellow PiDog enthusiasts!

I’m excited to share a project that has completely transformed my SunFounder PiDog from a simple servo-controlled robot into an interactive, personality-filled companion with real-time conversation abilities. Say hello to k9-polyvox!

What is k9-polyvox?

k9-polyvox is an open-source project that integrates OpenAI’s GPT-4o Realtime model with the PiDog platform to create a robot dog that can see, hear, speak, and express personality through movement. It’s like giving your PiDog a brain upgrade! There is no TTS or STT here, the AI processes audio directly in and out, so it’s very responsive and can sense things like sarcasm.

Why You’ll Love It

Here’s what makes k9-polyvox special:

:performing_arts: Multiple Personalities

The project includes 12 distinct personas, but these five are particularly noteworthy:

Vektor Pulsecheck: The system administrator persona that boots by default. This no-nonsense diagnostic assistant speaks in clipped, technical language and has exclusive access to system functions like shutdown commands. Perfect for checking battery status or running diagnostics with zero fluff.

Coach Chip Thunderbark: An INCREDIBLY ENTHUSIASTIC fitness coach who speaks in ALL CAPS and believes EVERY MOMENT is an opportunity for a MAXIMUM EFFORT WORKOUT!! Great for energizing a room or motivating kids to get moving.

Malvolio Dreadpaw: A dramatically sinister villain with delusions of grandeur who speaks with theatrical pauses and considers himself vastly superior to mere mortals. Children love his over-the-top evil persona.

Ember: A gentle, emotionally intelligent support robot designed specifically for comforting children. She excels at providing emotional support and helping users feel in control of difficult situations.

REX-4: A decommissioned military unit with glitchy transitions between rigid military protocols and newfound emotions. He sees potential threats everywhere while developing a personality beyond his programming.

You can also invent new personas on the fly, just ask the dog to create a new persona and provide a description.

:eye: Vision Capabilities

Your dog can actually SEE you! Ask it “What do you see?” and it will analyze its camera feed and describe what’s in front of it - according to its current persona’s perspective.

:clapper_board: Expressive Movements

The dog uses over 40 different physical actions (mostly from PiDog codebase) to express itself:

  • Head movements: look up/down/left/right, nod, shake head, think
  • Body actions: sit, stand, walk, turn, stretch, push-up
  • Emotional gestures: wag tail, bark, handshake, high five, pant

:studio_microphone: Real-Time Conversation

Natural back-and-forth conversations powered by OpenAI’s Realtime API. No wake word needed - just talk to your dog and it responds!

:bar_chart: Sensor Awareness

Your dog is aware of:

  • When it’s being petted (and responds accordingly!)
  • It’s visual surroundings
  • Which direction sounds come from
  • When people are in front of it
  • Its physical orientation
  • Battery level and system status

Try it out

The project is on GitHub at pgazmuri/k9-polyvox. I’d love to get feedback from the community!

Looking forward to your thoughts!

1 Like
2

Thank you. Looking forward to trying it out.

3

Thanks. Since the recent sudo level requirements and service patch (that I reported over on github) this works really well now. The grandkids love it!

Hope you don’t mind if I post a couple of short videos here? Truncated due to file size limit

4

Yes, please do… I haven’t had time to promote the project so appreciate the help! Be sure to ask Vektor to switch personas and try the other personalities. If you switch to “Malvolio” and ask him to roast you, it’s usually pretty funny.

5

I’ve tried a couple of the personalities, but not Malvolio. My batteries are really struggling at the moment to run for more than a few minutes, so it’s difficult for me to play around for very long.

As soon as I get the chance I’ll put some new higher capacity 18650’s in there.

I guess I could just disable the servos to get a bit more play time.

6

Hope I’m not intruding, but can someone please advise where I go to for help with K9-polydog? Thanks!

7

What is your specific problem?

I installed and ran it some time ago as per above videos. I don’t remember any installation problems, apart from those that I already posted on the authors github issues page, at his link above.

I don’t use it any more, mainly due to me using far too many chatgpt credits!

But post your error messages here, and someone may be able to help you, or post them formally on Pablo’s github page. The author, Pablo is the best source for help.

8

Yes I can help. Install can be tricky if you have messed with things like audio settings.

9

Thank you both for your replies.

I’ve carried out several complete clean installs to the letter (I think), but I seem to consistently get the same error when I try to start k9-polydog - “File “/usr/lib/python3/dist-packages/picamera2/picamera2.py”, line 365, in init
raise RuntimeError(“Camera init sequence did not complete.”)
RuntimeError: Camera init sequence did not complete.”

Not sure what to do about that. I was getting a different error to the effect of ‘openai’ not found, but I think I’ve fixed that by a fresh install of Python.

Thanks for any help.

10

Sorry to ask the obvious!

But does the camera work correctly for Sunfounder’s pidog examples?

Does it work from command line with libcamera-hello ?

Are you running bookworm? not an os-lite version?

11

Yes, the camera works exactly the same here (using libcamera) as it does with libcamera-hello and the other pidog examples.

I’m not sure what you mean by “from command line with libcamera-hello”… k9-polyvox cannot use the camera if another app is already using the camera, but you can launch k9-polyvox and libcamera-hello from the command line.

And yes this runs on the standard rpi4 bookworm Debian OS.

12

if I use my web browser and point it to http:/192.168.68.63:9000/mjpeg, then yes the camera works.

Can’t get this to work - cd ~/pidog/gpt_examples/
sudo ~/my_venv/bin/python3 gpt_dog.py

That gives me this - File “/home/pi/my_venv/lib/python3.11/site-packages/openai/_base_client.py”, line 1047, in request
raise self._make_status_error_from_response(err.response) from None
openai.AuthenticationError: Error code: 401 - {‘error’: {‘message’: ‘Missing bearer or basic authentication in header’, ‘type’: ‘invalid_request_error’, ‘param’: None, ‘code’: None}}

Don’t think I installed the lite version - how can I tell?

13

That’s gpt_dog, a different app from k9-polyvox. But it looks like you haven’t supplied your OpenAI API Key. You need to add it to a file in the gpt_dog directory if I remember correctly. I believe I setup k9-polyvox the same way, so it should be in “keys.py”.

14

Sorry Pablo. Those questions were meant for aeronut, not yourself. I was just trying to establish if their camera was connected correctly and/or functioning due to this error….

usr/lib/python3/dist-packages/picamera2/picamera2.py”, line 365, in init raise RuntimeError(“Camera init sequence did not complete.”)

However, as the web stream works it seems to be ok.

Apologies for any confusion!

15

OK, thanks for your dedication chaps!

I’ve been trying all sorts of things this morning and still getting the same problem - he’re the full error message -

[ActionManager] Using REAL preset actions.
vilib 0.3.17 launching …
picamera2 0.3.30
Camera init sequence did not complete.
Traceback (most recent call last):
File “/usr/lib/python3/dist-packages/picamera2/picamera2.py”, line 353, in init
self._open_camera()
File “/usr/lib/python3/dist-packages/picamera2/picamera2.py”, line 565, in _open_camera
self.camera.acquire()
RuntimeError: Failed to acquire camera: Device or resource busy

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File “/home/pi/k9-polyvox/main.py”, line 11, in
from action_manager import ActionManager
File “/home/pi/k9-polyvox/action_manager.py”, line 22, in
from t2_vision import TakePictureAndReportBack, is_person_detected, close_camera
File “/home/pi/k9-polyvox/t2_vision.py”, line 8, in
from vilib import Vilib
File “/usr/local/lib/python3.11/dist-packages/vilib/init.py”, line 2, in
from .vilib import Vilib
File “/usr/local/lib/python3.11/dist-packages/vilib/vilib.py”, line 182, in
class Vilib(object):
File “/usr/local/lib/python3.11/dist-packages/vilib/vilib.py”, line 184, in Vilib
picam2 = Picamera2()
^^^^^^^^^^^
File “/usr/lib/python3/dist-packages/picamera2/picamera2.py”, line 365, in init
raise RuntimeError(“Camera init sequence did not complete.”)
RuntimeError: Camera init sequence did not complete.

I’m baffled, so any light you’re able to shed on the problem is most welcome.

16

It looks like another process is using your camera. Do you have any boot/systemd etc startup scripts running? Or similar?

17

Not that I’ve done intentionally. How do I check?

18

You could try

systemctl list-units --type=service --state=running



 
 

However, if you've not added any other of your own work to this pidog, i.e. nothing you want to lose, then I'd suggest starting afresh. Something is awry and it's really difficult to debug remotely.'
19

OK, here’s the result of that -

UNIT LOAD ACTIVE SUB DESCRIPTION >
accounts-daemon.service loaded active running Accounts Service
avahi-daemon.service loaded active running Avahi mDNS/DNS-SD Stack
bluetooth.service loaded active running Bluetooth service
colord.service loaded active running Manage, Install and Generate >
cron.service loaded active running Regular background program pr>
cups-browsed.service loaded active running Make remote CUPS printers ava>
cups.service loaded active running CUPS Scheduler
dbus.service loaded active running D-Bus System Message Bus
getty@tty1.service loaded active running Getty on tty1
lightdm.service loaded active running Light Display Manager
ModemManager.service loaded active running Modem Manager
NetworkManager.service loaded active running Network Manager
packagekit.service loaded active running PackageKit Daemon
polkit.service loaded active running Authorization Manager
rtkit-daemon.service loaded active running RealtimeKit Scheduling Policy>
ssh.service loaded active running OpenBSD Secure Shell server
systemd-journald.service loaded active running Journal Service
systemd-logind.service loaded active running User Login Management
systemd-timesyncd.service loaded active running Network Time Synchronization
systemd-udevd.service loaded active running Rule-based Manager for Device>
triggerhappy.service loaded active running triggerhappy global hotkey da>
udisks2.service loaded active running Disk Manager

I see nothing there about the camera, but perhaps it means more to you than it does to me.

HOWEVER - this afternoon, i discovered that if I carry out a reset command before I start the main program, I no longer see the camera error. BUT I now get this error and I can’t find a way of stopping it -

Web display on:

http://192.168.68.66:9000/mjpg

Starting web streaming …

  • Serving Flask app ‘vilib.vilib’
  • Debug mode: off
    Camera Started
    Traceback (most recent call last):
    File “/home/pi/k9-polyvox/main.py”, line 11, in
    from action_manager import ActionManager
    File “/home/pi/k9-polyvox/action_manager.py”, line 25, in
    from display_manager import display_message, display_status
    File “/home/pi/k9-polyvox/display_manager.py”, line 4, in
    from luma.core.interface.serial import i2c
    ModuleNotFoundError: No module named ‘luma’
20

Did you run this from Pablo’s github page?

sudo pip install -r requirements.txt --break-system-packages