無線智能音箱實驗--使用Node-RED透過MQTT控制(Smart Wireless Speaker Experiment--Controlling by Node-RED via MQTT)

     本實驗製作一個WiFi音箱，使用著介面使用Node-RED Dashboard，透過MQTT protocol控制分布在不同地點的音箱，來播放音樂、網路收音機、Text to Speech廣播。可分別針對個別或全部音箱即時或排程播送。

    本實驗可應用於像校園廣播系統，可排程上下課鐘聲、或活動音樂分區播放、透過Node-RED dashboard介面輸入文字即可廣播語音。

一、實驗使用元件:

1. ESP32 NodeMCU-32S開發版
2. MAX98357A模組
3. RGB LED
4. NPN 電晶體
5. 3W4Ω喇叭
6. 200Ω電阻

二、開發軟體環境:

1. ESP-ADF(ESP-IDF)
2. VSCode
3. Node_RED
4. Mosquitto MQTT broker
5. SQLite

三、功能說明:

1. 即時顯示每個喇叭狀態(斷線、已連線或播放中)
2. Dashboard 能自動動態增加連線喇叭數量。
3. 設備端LED顯示狀態(LED紅燈為斷線、綠燈正常連線中)
4. 可針對各別或全部喇叭操作
5. 可即時播放音樂、網路收音機或TTS語音。
6. 可排程(月、日、時、分、星期)播放音樂、網路收音機或TTS語音。
7. MQTT使用TLS連線與帳密安全保護
8. 能使用HTTPS(secure HTTP) URI

四、實驗過程:

    1.定義MQTT TOPIC and message flows:

TOPIC分成兩大類，一為Node-RED送出控制Wifi Speaker動作，與Wifi Speaker回報狀態(STATE)給Node-RED，分別為/WIFI_SPEAKER/DeviceID/CMD 與 /WIFI_SPEAKER/DeviceID/STATE，DeviceID為每個WiFi Speaker  MAC Address 的後3bytes，用於區別對個別的音箱做控制與狀態顯示。詳細定義如下圖:

• CMD topic :
  TOPIC:/WIFI_SPEAKER/deviceID/CMD, 
  PAYLOAD: {"action":"play", "uri":"https://...../..mp3"}
  PAYLOAD: {"action":"stop"}
  PAYLOAD: {"action":"pause"}
  PAYLOAD: {"action":"resume"}
  PAYLOAD: {"action":"volume", "value":number}
• STATE topic:
  PAYLOAD: {"ID":"deviceID", "NAME":"deviceName", "TYPE":"NETWORK","STATE":"OFFLINE"}
  PAYLOAD: {"ID":"deviceID", "NAME":"deviceName", "TYPE":"NETWORK","STATE":"CONNECT"}
  PAYLOAD: {"ID":"deviceID",  "TYPE":"MUSIC_PLAY","STATE":"play"}
  PAYLOAD: {"ID":"deviceID",  "TYPE":"MUSIC_PLAY","STATE":"pause"}
  PAYLOAD: {"ID":"deviceID",  "TYPE":"MUSIC_PLAY","STATE":"resume"}
  PAYLOAD: {"ID":"deviceID",  "TYPE":"MUSIC_PLAY","STATE":"stop"}
  PAYLOAD: {"ID":"deviceID",  "TYPE":"MUSIC_PLAY","STATE":"FINISH"}
  
  
  兩個端點Node-RED與device(Wifi Speaker)對於接收到MQTT Topic要實行的端點動作如下圖所示:

2.ESP-ADF:

ESP-ADF (Espressif Audio Development Framework)，是ESP-IDF的擴充組件，用於開發Audio Application。在使用VSCode下，按下F1即可安裝ESP-ADF擴充組件，如下圖所示

有關ESP-ADF詳細文件說明位址:https://docs.espressif.com/projects/esp-adf/en/latest/。

本實驗以播放MP3格式的聲音為主，加入其他audio element codec即可播放其他格式audio，便於開發。

主要架構以audio pipeline透過ringbuffer串聯input audio element, process(codec) audio element與 output audio element，如下圖所示(取自官方文件)

本實驗I2S Output to DAC，不使用ESP32內建8bit DAC，使用MAX98357A 包含DAC與Ampiler。

3.TTS(Text To Speech)播放：

本實驗使用Google Translate TTS把Text轉成MP3語音於無線音箱播放。可使用兩種方式實行。

1. 安裝Node-RED google-translate-tts套件，將text轉成MP3 audio file存到server local file，再將local audio file的uri透過MQTT protocol送到WiFi Speaker播放，如下flow所示:
   
   
2. 安裝Node-RED node-red-contrib-google-tts套件，將欲播放的TEXT當成uri參數，透過node-red-contrib-google-tts轉成google translate URI，再透過MQTT protocol傳URI給WiFi Speaker播放。flow如下圖所示:
   
   

   
   
3. URI參數: a. tl=:為指定語言(en:英文, zh-TW中文)。 b. q=:為欲播放的語音文字。搜尋網路上有關google translate TTS的探討，URI亦可簡化類似:https://translate.google.com/translate_tts?ie=UTF-8&client=tw-ob&tl=en&ttsspeed=1&q=TEXT。

4.硬體線路:

• PIN_NUM_23連接npn電晶體與RGB LED用來指示Wifi連線MQTT是否正常，紅燈為斷線，綠燈為正常連線。
• PIN_NUM_5  I2S SCLK, PIN_NUM_25 I2S LRCK, PIN_NUM_26 I2S DSDIN

詳細線路圖如下所示：

4.Node-RED flows:

UI部分，本實驗主要使用Node-RED dashboard template(angular)來完成。

五、實驗成果展示

六、原始碼

parm_define.h

#define PRO_CPU         0
#define APP_CPU         1
#define SSID            "your-ssid"
#define PWD             "your-pwd"
#define MQTT_HOST       "your_mqtt_broker" 
#define MQTT_PORT       8883
#define MQTT_CLIENT_USER  "your_mqtt_user"
#define MQTT_CLIENT_PASSWORD  "your_mqtt_pwd"
#define DEVICE_NAME         "define_your_device_name"

static const char cert[] = "-----BEGIN CERTIFICATE-----\n"
"................................................................\n"
".....................\n"
"-----END CERTIFICATE-----";

static const uint8_t client_cert[]="-----BEGIN CERTIFICATE-----\n"
"................................................................\n"
"..................\n"
"-----END CERTIFICATE-----";

static const uint8_t client_key[] = "-----BEGIN RSA PRIVATE KEY-----\n"
"................................................................\n"
".....................\n"
"-----END RSA PRIVATE KEY-----";

wifi_speaker.c

#include <string.h>
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#include "freertos/event_groups.h"

#include "nvs_flash.h"
#include "audio_element.h"
#include "audio_pipeline.h"
#include "audio_event_iface.h"
#include "audio_common.h"
#include "http_stream.h"
#include "i2s_stream.h"
#include "mp3_decoder.h"
#include "driver/i2c.h"

#include "esp_log.h"
#include "esp_audio.h"
#include "esp_wifi.h"
#include "esp_peripherals.h"
#include "esp_netif.h"
#include "periph_wifi.h"
#include "esp_tls.h"
#include "driver/gpio.h"

#include "cJSON.h"
#include "mqtt_client.h"

#include "param_define.h" //wifi and mqtt connection parameters

static audio_event_iface_handle_t evt={0};
static audio_pipeline_handle_t pipeline;
static audio_element_handle_t http_stream_reader, i2s_stream_writer, mp3_decoder;
static esp_mqtt_client_handle_t hMQTTClient;
static char sID[7];
static char cmd_topic[50]={0};
static char state_topic[50]={0};

static esp_err_t mqtt_event_handler_cb(esp_mqtt_event_handle_t event)
{
    esp_mqtt_client_handle_t client = event->client;
    char msg[100]={0};
    char topic[100] = {0};
    switch (event->event_id) {
        case MQTT_EVENT_CONNECTED:
            sprintf(msg, "{\"ID\":\"%s\",\"NAME\":\"%s\",\"TYPE\":\"NETWORK\",\"STATE\":\"CONNECTED\"}", sID, DEVICE_NAME);
            esp_mqtt_client_publish(client, state_topic, msg, 0, 1, 0);
            esp_mqtt_client_subscribe(client, cmd_topic, 1);
            gpio_set_level(GPIO_NUM_23, 0);
            break;
        case MQTT_EVENT_DISCONNECTED:
            gpio_set_level(GPIO_NUM_23, 1);
            esp_mqtt_client_reconnect(client);
            break;
        case MQTT_EVENT_SUBSCRIBED:
            break;
        case MQTT_EVENT_UNSUBSCRIBED:
            break;
        case MQTT_EVENT_PUBLISHED:
            break;
        case MQTT_EVENT_DATA:
            strncpy(topic, event->topic, event->topic_len);
            if (strcmp(topic, cmd_topic) == 0) {
                cJSON *root = cJSON_Parse(event->data);
                char *action=cJSON_GetObjectItem(root, "action")->valuestring;
                if (strcmp(action,"play")==0) {
                    char *uri = cJSON_GetObjectItem(root, "uri")->valuestring;
                    int http_state = audio_element_get_state(http_stream_reader);
                    if (http_state == AEL_STATE_PAUSED) {
                        audio_pipeline_resume(pipeline);
                        while((http_state = audio_element_get_state(http_stream_reader)) != AEL_STATE_RUNNING) {
                            vTaskDelay(10/portTICK_PERIOD_MS);
                        }
                    }
                    audio_pipeline_stop(pipeline);
                    audio_pipeline_wait_for_stop(pipeline);
                    
                    audio_pipeline_reset_elements(pipeline);
                    audio_pipeline_reset_ringbuffer(pipeline);
                    audio_element_set_uri(http_stream_reader,uri);
                    
                    audio_pipeline_run(pipeline);
                    
                    sprintf(msg, "{\"ID\":\"%s\",\"TYPE\":\"MUSIC_PLAY\",\"STATE\":\"play\"}", sID);
                    esp_mqtt_client_publish(client,state_topic,msg,0,1,0);
                }
                if (strcmp(action,"stop")==0) {
                    audio_pipeline_stop(pipeline);
                    audio_pipeline_wait_for_stop(pipeline);
                    sprintf(msg, "{\"ID\":\"%s\",\"TYPE\":\"MUSIC_PLAY\",\"STATE\":\"stop\"}", sID);
                    esp_mqtt_client_publish(client,state_topic,msg,0,1,0);
                }
                if (strcmp(action,"pause")==0) {
                    audio_pipeline_pause(pipeline);
                    sprintf(msg, "{\"ID\":\"%s\",\"TYPE\":\"MUSIC_PLAY\",\"STATE\":\"pause\"}", sID);
                    esp_mqtt_client_publish(client,state_topic,msg,0,1,0);
                }
                if (strcmp(action,"resume")==0) {
                    audio_pipeline_resume(pipeline);
                    sprintf(msg, "{\"ID\":\"%s\",\"TYPE\":\"MUSIC_PLAY\",\"STATE\":\"resume\"}", sID);
                    esp_mqtt_client_publish(client,state_topic,msg,0,1,0);
                }
                if (strcmp(action,"volume")==0) {
                    int volume = cJSON_GetObjectItem(root, "value")->valueint;
                    i2s_alc_volume_set(i2s_stream_writer, volume);
                    sprintf(msg, "{\"ID\":\"%s\",\"TYPE\":\"MUSIC_PLAY\",\"STATE\":\"VOLUME\",\"volume\":%d}", sID, volume);
                    esp_mqtt_client_publish(client,state_topic,msg,0,1,0);
                }
            }
            break;
        case MQTT_EVENT_ERROR:
            if (event->error_handle->error_type == MQTT_ERROR_TYPE_TCP_TRANSPORT) {
            }
            break;
        default:
            break;
    }
    return ESP_OK;
}

static void mqtt_event_handler(void *handler_args, esp_event_base_t base, int32_t event_id, void *event_data) {
    mqtt_event_handler_cb(event_data);
}

static void mqtt_client_start(void)
{
    char msg[100];
    sprintf(msg, "{\"ID\":\"%s\",\"NAME\":\"%s\",\"TYPE\":\"NETWORK\",\"STATE\":\"OFFLINE\"}", sID, DEVICE_NAME);
    esp_mqtt_client_config_t mqtt_cfg = {
        .host = MQTT_HOST, 
        .port = MQTT_PORT,
        //.uri = MQTT_URI,
        .username = MQTT_CLIENT_USER,
        .password = MQTT_CLIENT_PASSWORD,
        .lwt_topic = state_topic,
        .lwt_msg = msg,
        .keepalive=30,
        .cert_pem=(const char *)cert,
        .cert_len=0,
        .client_cert_pem=(const char *)client_cert,
        .client_cert_len=0,
        .client_key_pem=(const char *)client_key,
        .client_key_len=0,
        .transport = MQTT_TRANSPORT_OVER_SSL
        
    };
    hMQTTClient = esp_mqtt_client_init(&mqtt_cfg);
    esp_mqtt_client_register_event(hMQTTClient, ESP_EVENT_ANY_ID, mqtt_event_handler, hMQTTClient);
    esp_mqtt_client_start(hMQTTClient);
}

void wifi_speaker_audio_pipeline_init(void) {
  
    audio_pipeline_cfg_t pipeline_cfg = DEFAULT_AUDIO_PIPELINE_CONFIG();
    //pipeline_cfg.rb_size=40960;
    pipeline = audio_pipeline_init(&pipeline_cfg);
    mem_assert(pipeline);

    // audio element http_stream
    http_stream_cfg_t http_cfg = HTTP_STREAM_CFG_DEFAULT();
    http_cfg.type = AUDIO_STREAM_READER;
     http_cfg.task_core = APP_CPU;
    http_stream_reader = http_stream_init(&http_cfg);

    //audio element i2s_stream
    i2s_stream_cfg_t i2s_cfg = I2S_STREAM_CFG_DEFAULT();
    i2s_cfg.task_core = APP_CPU;
    i2s_cfg.use_alc=true;
    i2s_cfg.type = AUDIO_STREAM_WRITER;
    i2s_stream_writer = i2s_stream_init(&i2s_cfg);
   
    // audio element mp3_decoder
    mp3_decoder_cfg_t mp3_cfg = DEFAULT_MP3_DECODER_CONFIG();
    mp3_cfg.task_core = APP_CPU;
    mp3_decoder = mp3_decoder_init(&mp3_cfg);

    // register audio element in the order http->mp3->i2s(input -> process -> output)
    audio_pipeline_register(pipeline, http_stream_reader, "http");
    audio_pipeline_register(pipeline, mp3_decoder,        "mp3");
    audio_pipeline_register(pipeline, i2s_stream_writer,  "i2s");
    
    // Link it together http_stream-->mp3_decoder-->i2s_stream-->[MAX98357A]");
    const char *link_tag[3] = {"http", "mp3", "i2s"};
     audio_pipeline_link(pipeline, &link_tag[0], 3);
    
}

void app_main(void)
{
    //led turn off
    gpio_pad_select_gpio(GPIO_NUM_23);
    gpio_set_direction(GPIO_NUM_23,GPIO_MODE_OUTPUT);
    gpio_set_level(GPIO_NUM_23, 1);
    esp_err_t err = nvs_flash_init();
    if (err == ESP_ERR_NVS_NO_FREE_PAGES) {
        // NVS partition was truncated and needs to be erased
        // Retry nvs_flash_init
        ESP_ERROR_CHECK(nvs_flash_erase());
        err = nvs_flash_init();
    }
    
    ESP_ERROR_CHECK(esp_netif_init());

    //esp_log_level_set("*", ESP_LOG_WARN);
    //esp_log_level_set(TAG, ESP_LOG_DEBUG);

    wifi_speaker_audio_pipeline_init();


    esp_periph_config_t periph_cfg = DEFAULT_ESP_PERIPH_SET_CONFIG();
    esp_periph_set_handle_t set = esp_periph_set_init(&periph_cfg);
    periph_wifi_cfg_t wifi_cfg = {
        .ssid = SSID,
        .password = PWD,
        
    };
    esp_periph_handle_t wifi_handle = periph_wifi_init(&wifi_cfg);
    esp_periph_start(set, wifi_handle);
    esp_err_t cr = periph_wifi_wait_for_connected(wifi_handle, 120000/portTICK_PERIOD_MS);
    if (cr == ESP_FAIL) {
        return;
    }

    // register audio event
    audio_event_iface_cfg_t evt_cfg = AUDIO_EVENT_IFACE_DEFAULT_CFG();
    evt = audio_event_iface_init(&evt_cfg);
    // Listening event from all elements of pipeline
    audio_pipeline_set_listener(pipeline, evt);
    // Listening event from peripherals
    audio_event_iface_set_listener(esp_periph_set_get_event_iface(set), evt);
    uint8_t tmac[6];
    esp_efuse_mac_get_default(tmac);
    sprintf(sID, "%02x%02x%02x", tmac[3],tmac[4],tmac[5]);
    sprintf(cmd_topic, "/WIFI_SPEAKER/%s/CMD", sID);
    sprintf(state_topic, "/WIFI_SPEAKER/%s/STATE", sID);
    
    // default volume level  (range -64 ~ 64)
    i2s_alc_volume_set(i2s_stream_writer, -20);
    
    mqtt_client_start();

    while(1) {
        audio_event_iface_msg_t msg;
        esp_err_t ret = audio_event_iface_listen(evt, &msg, portMAX_DELAY);
        if (ret != ESP_OK) {
            continue;
        }

        if (msg.source_type == AUDIO_ELEMENT_TYPE_ELEMENT
            && msg.source == (void *) mp3_decoder
            && msg.cmd == AEL_MSG_CMD_REPORT_MUSIC_INFO) {
            audio_element_info_t music_info = {0};
            audio_element_getinfo(mp3_decoder, &music_info);
            audio_element_setinfo(i2s_stream_writer, &music_info);
            i2s_stream_set_clk(i2s_stream_writer, music_info.sample_rates, music_info.bits, music_info.channels);
            continue;
        }
        if (msg.source_type == AUDIO_ELEMENT_TYPE_ELEMENT
            && msg.source == (void *) i2s_stream_writer
            && msg.cmd == AEL_MSG_CMD_REPORT_STATUS && (int)msg.data == AEL_STATUS_STATE_FINISHED) {
                char mqttdata[100];
                sprintf(mqttdata, "{\"ID\":\"%s\",\"TYPE\":\"MUSIC_PLAY\",\"STATE\":\"FINISH\"}", sID);
                esp_mqtt_client_publish(hMQTTClient,state_topic,mqttdata,0,1,0);
            
            continue;
        }
        if (msg.source_type == AUDIO_ELEMENT_TYPE_ELEMENT
            && msg.source == (void *) http_stream_reader
            && msg.cmd == AEL_MSG_CMD_REPORT_STATUS && (int)msg.data == AEL_STATUS_ERROR_OPEN) {
                char mqttdata[100];
                sprintf(mqttdata, "{\"ID\":\"%s\",\"TYPE\":\"MUSIC_PLAY\",\"STATE\":\"openerror\"}", sID);
                esp_mqtt_client_publish(hMQTTClient,state_topic,mqttdata,0,1,0);
            
            continue;
        }
    }
}