xiaozhi-esp32

An MCP-based Chatbot

(English | 中文 | 日本語)

Introduction

👉 Human: Give AI a camera vs AI: Instantly finds out the owner hasn't washed hair for three days【bilibili】

👉 Handcraft your AI girlfriend, beginner's guide【bilibili】

As a voice interaction entry, the XiaoZhi AI chatbot leverages the AI capabilities of large models like Qwen / DeepSeek, and achieves multi-terminal control via the MCP protocol.

Recent Updates

• The project now requires ESP-IDF v6.0.1 or later. ESP-IDF release/v6.1 is the recommended SDK branch. ESP-IDF 5.x is no longer supported. The current matrix contains 171 variants; ESP32-S31 builds require IDF 6.1 or later.

• MQTT and BluFi cryptographic code has migrated to PSA Crypto. IDF 6 component splits and third-party dependency compatibility have also been addressed.

• Audio pipeline concurrency, MQTT/UDP packet validation, and release-matrix selection have been hardened.

• ESP32-P4 Rev1 and Rev3 are both supported on IDF 6 with ESP-SR 2.4.7.

Features Implemented

• Wi-Fi, wired Ethernet, USB RNDIS, and ML307/EC801E or NT26 Cat.1 4G networking; supported boards can switch between Wi-Fi and 4G

• Offline voice wake-up with ESP-SR, including customizable wake words

• Two communication transports: WebSocket and MQTT + UDP

• Opus audio streaming with conventional streaming ASR + LLM + TTS pipelines and Realtime end-to-end voice models; AEC-capable hardware supports realtime full-duplex interaction

• Speaker recognition, identifies the current speaker 3D Speaker

• OLED / LCD displays with emoji and rich expression support, plus camera vision input on supported boards

• Battery display and power management

• 39 interface languages, with localized voice prompts where available and English fallback

• ESP32, ESP32-C3, ESP32-C5, ESP32-C6, ESP32-S3, and ESP32-P4 chip platforms

• Wi-Fi provisioning through hotspot or BluFi

• Device-side MCP for device control (Speaker, LED, Servo, GPIO, etc.)

• Cloud-side MCP to extend large model capabilities (smart home control, PC desktop operation, knowledge search, email, etc.)

• Customizable wake words, fonts, emojis, and chat backgrounds with online web-based editing (Custom Assets Generator)

Hardware

Breadboard DIY Practice

See the Feishu document tutorial:

👉 "XiaoZhi AI Chatbot Encyclopedia"

Breadboard demo:

Supports 138 Board Directories and 171 Release Variants (Partial List)

• LiChuang ESP32-S3 Development Board

• Espressif ESP32-S3-BOX-3

• M5Stack CoreS3

• M5Stack AtomS3R + Echo Base

• Magic Button 2.4

• Waveshare ESP32-S3-Touch-AMOLED-1.8

• LILYGO T-Circle-S3

• XiaGe Mini C3

• CuiCan AI Pendant

• WMnologo-Xingzhi-1.54TFT

• SenseCAP Watcher

• ESP-HI Low Cost Robot Dog

Software

Firmware Flashing

For beginners, it is recommended to use the firmware that can be flashed without setting up a development environment.

The firmware connects to the official xiaozhi.me server by default. Personal users can register an account to use the Qwen real-time model for free.

👉 Beginner's Firmware Flashing Guide

Development Environment

• Cursor or VSCode

• Install the ESP-IDF plugin. The minimum SDK is ESP-IDF v6.0.1; ESP-IDF release/v6.1 is recommended. ESP-IDF 5.x is not supported.

• Linux is better than Windows for faster compilation and fewer driver issues

• This project uses Google C++ code style, please ensure compliance when submitting code

Use the release/v6.1 branch rather than the v6.1 release tag: an ESP32-C3 Wi-Fi initialization failure with that tag is reported in #2344. After switching SDK revisions, run git submodule update --init --recursive in the ESP-IDF directory, source its export.sh, and run idf.py fullclean in this project before rebuilding.

Developer Documentation

• Custom Board Guide - Learn how to create custom boards for XiaoZhi AI

• MCP Protocol IoT Control Usage - Learn how to control IoT devices via MCP protocol

• MCP Protocol Interaction Flow - Device-side MCP protocol implementation

• MQTT + UDP Hybrid Communication Protocol Document

• A detailed WebSocket communication protocol document

Large Model Configuration

If you already have a XiaoZhi AI chatbot device and have connected to the official server, you can log in to the xiaozhi.me console for configuration.

👉 Backend Operation Video Tutorial (Old Interface)

Related Open Source Projects

For server deployment on personal computers, refer to the following open-source projects:

• xinnan-tech/xiaozhi-esp32-server Python server

• joey-zhou/xiaozhi-esp32-server-java Java server

• AnimeAIChat/xiaozhi-server-go Golang server

• hackers365/xiaozhi-esp32-server-golang Golang server

Other client projects using the XiaoZhi communication protocol:

• huangjunsen0406/py-xiaozhi Python client

• TOM88812/xiaozhi-android-client Android client

• 100askTeam/xiaozhi-linux Linux client by 100ask

• 78/xiaozhi-sf32 Bluetooth chip firmware by Sichuan

• QuecPython/solution-xiaozhiAI QuecPython firmware by Quectel

Custom Assets Tools:

• 78/xiaozhi-assets-generator Custom Assets Generator (Wake words, fonts, emojis, backgrounds)

About the Project

This is an open-source ESP32 project, released under the MIT license, allowing anyone to use it for free, including for commercial purposes.