EPS32 ESP-IDF開發環境儲存設備與檔案系統實驗(二) -- SPI FLASH (A Study on ESP32 Storage Devices and File System--SPI Flash using ESP-IDF)

本實驗是在ESP-IDF環境下，測試ESP32 spi flash storage。使用內建(main flash)與外接flash(external falsh) ，分別測試nvs, spiffs 與fat filesystem。

本實驗使用的外接flash為Winbond W25Q128FV外接模組，

共有VCC, GND, CS, CLK, DI 與DO pins，IO2與IO3未接出。根據W25Q128FV Datasheeet只能操作在standard spi 與dual spi operation。pin No.相對應表如下圖。

(取自Winbond W25Q128FV datasheet)

ESP32 ESP-IDF環境下直接支援Winbond flash，memuconfig預設是開啟支援的

一、加入 SPI Flash device 

• 內建main flash在memuconfig 中指定mode, speed等，例如QIO, 80MHz

• 外接SPI Flash依序以下列三個API就能加入系統中

1. spi_bus_initialize(): 決定使用哪一個SPI HOST，MISO、MOSI、CLK pin和 DMA channel.
2. spi_bus_add_flash_device():  附加 flash device 在SPI bus上.  指定CS pin, I/O Mode(dio、qio) 與spi bus speed等。
3. esp_flash_init(): 開始與flash溝通. 偵測chip型號等。

三個API參數詳細說明如下：

esp_err_t spi_bus_initialize(spi_host_device_t host_id, const spi_bus_config_t * bus_config, spi_dma_chan_t dma_chan)

1. host_id:SPI1_HOST, SPI2_HOST(HSPI_HOST), SPI3_HOST(VSPI_HOST)。例如
   VSPI_HOST。
2. spi_bus_config_t: 指定miso_io_num, mosi_io_num and sclk_io_num，本實驗例子spi_bus_config_t bus_config = {
           .mosi_io_num = GPIO_NUM_23,
           .miso_io_num = GPIO_NUM_19,
           .sclk_io_num = GPIO_NUM_18 
       };
3. spi_dma_chan_t: DMA channel，例如
   spi_dma_chan_t spi_dma_chan = SPI_DMA_CH2;

esp_err_t spi_bus_add_flash_device(esp_flash_t ** out_chip, const esp_flash_spi_device_config_t * config)

1. esp_flash_t: 代表此flash device的pointer，例如
   esp_flash_t * flash_chip;
2. esp_flash_spi_device_config_t: 指定此flash device的參數，io_mode, speed, cs pin。例如esp_flash_spi_device_config_t spi_device_config = {
           .host_id = VSPI_HOST,
           .cs_id = 0,
           .cs_io_num = 5,
           .io_mode = SPI_FLASH_DIO,
           .speed = ESP_FLASH_40MHZ
       };

esp_err_t IRAM_ATTR esp_flash_init(esp_flash_t * chip);

經過這三個API設定，順利加入spi flash device後，代表此flash device的pointer(上例為:flash_chip)即可以此flash_chip作為後續API的參數。測試系統順利附加spi flash device，如下圖。

(執行結果截圖)

二、分割SPI Flash Device Partition

flash 的partition內容分為幾個部分:

1. label: 代表此partition名稱。
2. type: app or data。
3. subtype: app 或data的subtype，本實驗操作的三個檔案系統為type是data的subtype分別為nvs, spiffs, fat
4. offset: 位移位址
5. size:大小
6. flags: 指定是否encrypted

• 內建flash(main flash) partition建立方式:

內建flash(main flash) 的partition table可在menuconfig中指定使用Custom partition CSV，如下圖。

製作csv檔案，build project即可為內建flash建立所需的partitions，例如

• 外接SPI flash partition table建立方式:

使用API esp_partition_register_external來將外接SPI Flash指定每個partition 的label, type, subtype, offset,size等參數，將partition table加入的flash_chip中。
此API詳細參數說明如下:
esp_err_t esp_partition_register_external(esp_flash_t * flash_chip, size_t offset, size_t size, const char * label, esp_partition_type_t type, esp_partition_subtype_t subtype, const esp_partition_t ** out_partition)

1. esp_flash_t: 上述spi_bus_add_flash_device API, return 的 flshp_chip
2. offset: 指定 offset
3. size: 指定大小
4. esp_partition_type_t: 例如ESP_PARTITION_TYPE_DATA為data type
5. esp_partition_subtype_t:例如 ESP_PARTITION_SUBTYPE_DATA_FAT為fat subtype,
6. esp_partiton_t: 為output partition pointer作為後續處理partition用，例如本實驗中fat_part, nvs_part等。

本實驗partition的lable, offset, size分別如下列設定:

const size_t fatOffset = 0, fatSize=8*1024*1024; // 8MB, use for FAT partition

const size_t nvsOffset = 8*1024*1024, nvsSize = 1024*1024; // 1MB, use for nvs partition

const size_t spiffsOffset = 9*1024*1024, spiffsSize = 2*1024*1024; // 2M, user for spiffs partition

const esp_partition_t *fat_part;

const esp_partition_t *nvs_part;

const esp_partition_t *spiffs_part;

// partition label

const char extern_flash_fat[] = "extFATData";

const char extern_flash_nvs[] = "extNVS";

const char extern_flash_spiffs[] = "extSPIFFS";

三、掛載SPI Flash Device Partition

本實驗使用nvs, spiffs, fat 三種 flash partitions，各別partition type說明如下:

NVS partition

    Non-volatile storage (NVS) 常用來儲存 key-value 資料在flash中。partition 的type為data, subtype 為nvs，當partition label 為 nvs 稱為default NVS partition。

    首先需初始化，nvs_flash_init(void)初始化default nvs partiton，使用nvs_flash_init_partiton(const char *partition_label)初始化其他nvs parttiton，例如nvs_flash_init_partiton("extNVS");

其他常用使令:

1. nvs_flash_erase，nvs_flash_erase_partition:抹除整個partition。
2. nvs_open,  nvs_open_from_partition: 開啟namespace, readonly or readwrite mode
3. nvs_set_*: 設定key-value, value可為string, blob , integer or usigned integer。
4. nvs_get_*: 取得key的value
5. nvs_commit:寫入到flash中
6. nvs_close:關閉並釋放資源

SPIFFS partition 

Flash的區塊有erase/write次數的限制，同一個區塊重複erase/write，將會造成讀取速度變慢，甚至損壞。使用Wear Leveling(耗損平均技術)能平均使用快閃記憶體中的每個儲存區塊，來避免某些儲存區塊因過度使用而形成壞區塊。

SPIFFS is a file system 常使用在內鑲系統 SPI NOR flash 設備上. 支援 Wear Leveling, file system 一致性檢查, 但不支援目錄架構。

使用 esp_vfs_spiffs_register掛載partition在vfs後，即可使用標準C file library處理spiffs檔案
esp_err_t esp_vfs_spiffs_register(const esp_vfs_spiffs_conf_t * conf)

參數esp_vfs_spiffs_conf_t指定掛載base path, partition,最大可開啟檔案數，掛載失敗是否要格式化等，例如:

esp_vfs_spiffs_conf_t spiffsConf = {

        .base_path="/spiffs",

        .partition_label = "extSPIFFS", 

        .max_files=5,

        .format_if_mount_failed=true

    };

當掛在extSPIFFS partition的base path為"/spiffs"時，若要開啟"test.txt" 檔案，以指令fopen("/spiffs/test.txt","w");開啟謝入模式。

esp_vfs_spiffs_unregister:從VFS卸載SPIFFS。

FAT partition

使用FAT filesystem在Flash上與使用在SD Card上類似，但增加Wear Levelling設定。

常用指令:

esp_vfs_fat_spiflash_mount, esp_vfs_fat_spiflash_unmount, 

掛載 partition API:

esp_err_t esp_vfs_fat_spiflash_mount(const char *base_path, const char *partition_label, const esp_vfs_fat_mount_config_t *mount_config, wl_handle_t *wl_handle)

ESP-IDF 使用此單一API，完成下堆堆疊動作:把partition table中相對應的partition label做Wear Levelling library初始化，使用FATFS library掛載FAT partition在wear liveiling libarary上，最後註冊VFS的base path，之後即以standard C file libaray存取flash device上的檔案。

API參數說明如下:

1. base_path: 為掛載partition label的base path，例如:"/spiflash"。
2. partition label: 欲掛載的 partition label，例如:"extFATData":
3. esp_vsf_fat_mount_config_t: 說明掛載失敗是否格式化，最大可開啟檔案數等，例如esp_vfs_fat_mount_config_t spiflash_mount_config = {
           .format_if_mount_failed = true,
           .max_files=3,
           .allocation_unit_size=CONFIG_WL_SECTOR_SIZE
       };
4. wl_handle_t: wear levelling handle，作為後續其他API使用的參數，例如:
   static wl_handle_t wl_handle = WL_INVALID_HANDLE;

卸載partition API:

esp_err_t esp_vfs_fat_spiflash_unmount(const char * base_path, wl_handle_t wl_handle);

四、實驗結果

第一次使用flsah device，尚未分割partition與格式化， spiffs與fat filesystem根據esp_vfs_fat_mount_t指定.format_if_mount_failed = true,自動格式化該partition，如下圖所

分別在內建flash(main flash)與外加flash模組開啟nvs, spiffs 與 FAT partition，個別執行read/write，結果如下圖所示:

五、程式碼

#include <stdio.h>
#include "freertos/FreeRTOS.h"
#include "freertos/task.h"
#include "esp_err.h"
#include "esp_log.h"
#include "driver/sdspi_host.h"
#include "esp_partition.h"
#include "esp_spi_flash.h"
#include "esp_spiffs.h"
#include "esp_flash.h"
#include "esp_vfs_fat.h"
#include "esp_flash_spi_init.h"
#include "nvs_flash.h"
#include "esp_spiffs.h"

#define TAG "SPI FLASH TEST"

static esp_flash_t *flash_chip;
static wl_handle_t wl_handle = WL_INVALID_HANDLE;

const char extern_flash_fat[] = "extFATData";
const char extern_flash_nvs[] = "extNVS";
const char extern_flash_spiffs[] = "extSPIFFS";
const size_t fatOffset = 0, fatSize=8*1024*1024; // 8MB
const size_t nvsOffset = 8*1024*1024, nvsSize = 1024*1024; // 1MB
const size_t spiffsOffset = 9*1024*1024, spiffsSize = 2*1024*1024; // 2M


    
const esp_partition_t *fat_part;
const esp_partition_t *nvs_part;
const esp_partition_t *spiffs_part;

bool external_flash_init() {
    esp_err_t ret;
 
    spi_host_device_t vspi_host_id = VSPI_HOST;
    spi_bus_config_t bus_config = {
        .mosi_io_num = GPIO_NUM_23,
        .miso_io_num = GPIO_NUM_19,
        .sclk_io_num = GPIO_NUM_18  
    };
    spi_dma_chan_t spi_dma_chan = SPI_DMA_CH2;
    //step 1 spi_bus_initialize
    ret = spi_bus_initialize(vspi_host_id, &bus_config, spi_dma_chan);
    if (ret != ESP_OK) {
        ESP_LOGI(TAG, "spi bus initialize failure: %s", esp_err_to_name(ret));
        return false;
    }
    // step 2 spi_bus_add_device
    
    esp_flash_spi_device_config_t spi_device_config = {
        .host_id = VSPI_HOST,
        //.cs_id = 0,
        .cs_io_num = 5,
        //.io_mode = SPI_FLASH_QIO,
        .io_mode = SPI_FLASH_DIO,
        .speed = ESP_FLASH_40MHZ
    };
    ret = spi_bus_add_flash_device(&flash_chip, &spi_device_config);
    if (ret != ESP_OK) {
        ESP_LOGI(TAG, "spi bus add flash device failure: %s", esp_err_to_name(ret));
        return false;
    }
    //step 3 spi_flash_init
    ret = esp_flash_init(flash_chip);
    if (ret != ESP_OK) {
        ESP_LOGI(TAG, "spi flash init: %s", esp_err_to_name(ret));
        return false;
    }
    uint32_t fsize;
    ret = esp_flash_get_size(flash_chip, &fsize);
    ESP_LOGI(TAG, "external SPI Flash(W25Q128FV) size:0x%0x, chip size:0x%0x", fsize, flash_chip->size);

    return true;
}

bool register_external_partition() {
    esp_err_t ret;
    
    ret = esp_partition_register_external(flash_chip, fatOffset, fatSize, extern_flash_fat, ESP_PARTITION_TYPE_DATA, ESP_PARTITION_SUBTYPE_DATA_FAT, &fat_part);
    if (ret != ESP_OK) {
        ESP_LOGI(TAG, "register external fat partitional error:%s", esp_err_to_name(ret));
        return false;
    }
    
    ret = esp_partition_register_external(flash_chip, nvsOffset, nvsSize, extern_flash_nvs, ESP_PARTITION_TYPE_DATA, ESP_PARTITION_SUBTYPE_DATA_NVS, &nvs_part);
    if (ret != ESP_OK) {
        ESP_LOGI(TAG, "register external nvs partitional error:%s", esp_err_to_name(ret));
        return false;
    }
    
    ret = esp_partition_register_external(flash_chip, spiffsOffset, spiffsSize, extern_flash_spiffs, ESP_PARTITION_TYPE_DATA, ESP_PARTITION_SUBTYPE_DATA_SPIFFS, &spiffs_part);
    if (ret != ESP_OK) {
        ESP_LOGI(TAG, "register external spiffs partitional error:%s", esp_err_to_name(ret));
        return false;
    }

    return true;
}

void nvs_read_write_test() {
    esp_err_t ret;

    //test main flash default nvs partition
    ESP_LOGI(TAG, "TEST main flash default nvs partition read/write");
    ret = nvs_flash_init();
    if (ret == ESP_ERR_NVS_NEW_VERSION_FOUND || ret == ESP_ERR_NVS_NO_FREE_PAGES) {
        nvs_flash_erase();
        nvs_flash_init();
    }
    nvs_handle_t hNVS;
    ret = nvs_open("defaultnvs", NVS_READWRITE, &hNVS);
    if (ret != ESP_OK) {
        ESP_LOGI(TAG, "open default nvs error:%s", esp_err_to_name(ret));
        return;
    }
    nvs_set_str(hNVS, "testK1", "default Key1 value");
    nvs_set_str(hNVS, "testK2", "default Key2 value");
    nvs_commit(hNVS);
    nvs_close(hNVS);

    ret = nvs_open("defaultnvs", NVS_READWRITE, &hNVS);
    if (ret != ESP_OK) {
        ESP_LOGI(TAG, "open default nvs error:%s", esp_err_to_name(ret));
        return;
    }
    char tvalue[100];
    size_t tsize=100;
    nvs_get_str(hNVS, "testK1", tvalue, &tsize);
    printf("Read default nvs testK1, value=%s\n", tvalue);
    tsize=100;
    nvs_get_str(hNVS, "testK2", tvalue, &tsize);
    printf("Read default nvs testK1, value=%s\n\n", tvalue);
    nvs_close(hNVS);


    ESP_LOGI(TAG, "TEST external flash nvs_part partition read/write");
    ret = nvs_flash_init_partition(extern_flash_nvs);
    if (ret == ESP_ERR_NVS_NEW_VERSION_FOUND || ret == ESP_ERR_NVS_NO_FREE_PAGES) {
        nvs_flash_erase_partition(extern_flash_nvs);
        nvs_flash_init_partition(extern_flash_nvs);
    }
   
    ret = nvs_open_from_partition(extern_flash_nvs, "externalnvs", NVS_READWRITE, &hNVS);
    if (ret != ESP_OK) {
        ESP_LOGI(TAG, "open external flash nvs error:%s", esp_err_to_name(ret));
        return;
    }
    nvs_set_str(hNVS, "extK1", "external flash nvs Key1 value");
    nvs_set_str(hNVS, "extK2", "external flash nvs Key2 value");
    nvs_commit(hNVS);
    nvs_close(hNVS);
    printf("\n");
    ret = nvs_open_from_partition(extern_flash_nvs, "externalnvs", NVS_READWRITE, &hNVS);
    if (ret != ESP_OK) {
        ESP_LOGI(TAG, "open external flash nvs error:%s", esp_err_to_name(ret));
        return;
    }

    nvs_get_str(hNVS, "extK1", tvalue, &tsize);
    printf("Read external flash nvs extK1, value=%s\n", tvalue);
    tsize=100;
    nvs_get_str(hNVS, "extK2", tvalue, &tsize);
    printf("Read external flash nvs extK1, value=%s\n\n", tvalue);
    nvs_close(hNVS);
    printf("\n");
}

void spiffs_read_write_test() {
    esp_err_t ret;
    size_t t,u;
    FILE *fp;
    char buf[100];
   
    esp_vfs_spiffs_conf_t spiffsConf = {
        .base_path="/spiffs",
        .partition_label = "mainspiffs", 
        .max_files=5,
        .format_if_mount_failed=true
    };
    
    // test main flash spiffs 
    ESP_LOGI(TAG, "main flash spiffs read/write test");
    ret = esp_vfs_spiffs_register(&spiffsConf);
    if (ret != ESP_OK) {
        ESP_LOGI(TAG,"mainflash spiffs register error:%s\n",esp_err_to_name(ret));
        return;
    }
    esp_spiffs_info("mainspiffs", &t, &u);
    ESP_LOGI(TAG, "main flash spiffs size -- Total:%d, used:%d\n", t,u);

    fp = fopen("/spiffs/mtest.txt", "w");
    if (fp) {
        fputs("main flash spiffs test line 1\n", fp);
        fputs("main flash spiffs test line 2\n", fp);
        fputs("main flash spiffs test line 3\n", fp);
        fclose(fp);
    } else {
        ESP_LOGI(TAG, "open main flash spiff error(write)");
        return;
    }

    
    fp = fopen("/spiffs/mtest.txt", "r");
    if (!fp) {
        ESP_LOGI(TAG, "open main flash spiff error(read)");
        return;
    }
    while(fgets(buf, 100, fp)) {
        printf("read from main flash spiffs: %s",buf);
    }
    fclose(fp);
    printf("\n");
    esp_vfs_spiffs_unregister("mainspiffs");

    // test external flash spiffs
    esp_vfs_spiffs_conf_t extspiffsConf = {
        .base_path="/spiffs",
        .partition_label = extern_flash_spiffs, 
        .max_files=5,
        .format_if_mount_failed=true
    };

    ESP_LOGI(TAG, "external flash spiffs read/write test");
    ret = esp_vfs_spiffs_register(&extspiffsConf);
    if (ret != ESP_OK) {
        ESP_LOGI(TAG,"external flash spiffs register error: %s\n",esp_err_to_name(ret));
        return;
    }
    esp_spiffs_info(extern_flash_spiffs, &t, &u);
    ESP_LOGI(TAG, "external flash spiffs size -- Total:%d, used:%d\n", t,u);

    fp = fopen("/spiffs/mtest.txt", "w");
    if (fp) {
        fputs("external flash spiffs test line 1\n", fp);
        fputs("external flash spiffs test line 2\n", fp);
        fputs("external flash spiffs test line 3\n", fp);
        fclose(fp);
    }

    fp = fopen("/spiffs/mtest.txt", "r");
    while(fgets(buf, 100, fp)) {
        printf("read from external flash spiffs ----- %s",buf);
    }
    fclose(fp);
    printf("\n");
   esp_vfs_spiffs_unregister(extern_flash_spiffs);
    
}

void fat_read_write_test() {
    esp_err_t ret;
    FILE *fp;
    char buf[100];

     esp_vfs_fat_mount_config_t spiflash_mount_config = {
        .format_if_mount_failed = true,
        .max_files=3,
        .allocation_unit_size=CONFIG_WL_SECTOR_SIZE
    };
// test main spi flash fat filesysetm
    ESP_LOGI(TAG, "test main spi flash fat filesystem");
    ret = esp_vfs_fat_spiflash_mount("/spiflash", "mainfat", &spiflash_mount_config, &wl_handle);
    if (ret != ESP_OK) {
        ESP_LOGI(TAG, "mount main flash fat filesystem error: %s", esp_err_to_name(ret));
        return;
    }
    fp = fopen("/spiflash/ftest.txt", "wb");
    if (fp) {
        fputs("main spi flash fat filesystem test, LINE 1\n", fp);
        fputs("main spi flash fat filesystem test, LINE 2\n", fp);
        fputs("main spi flash fat filesystem test, LINE 3\n", fp);
        fclose(fp);
    } else {
        ESP_LOGI(TAG, "open main flash write fat file error(write)\n");
        return;
    }
    ESP_LOGI(TAG, "main spi flash write finished");
    fp = fopen("/spiflash/ftest.txt", "rb");
    if(fp) {
        while(fgets(buf, 100, fp)) {
            printf(buf);
        }
        fclose(fp);
        printf("\n");
    }else  {
         ESP_LOGI(TAG, "open main flash write fat file error(read)\n");
         return;
    }

    esp_vfs_fat_spiflash_unmount("/spiflash", wl_handle);
// test external spi flash fat filesysetm
    ESP_LOGI(TAG, "test external spi flash fat filesystem");
    ret = esp_vfs_fat_spiflash_mount("/spiflash", extern_flash_fat, &spiflash_mount_config, &wl_handle);
    if (ret != ESP_OK) {
        ESP_LOGI(TAG, "mount external flash fat filesystem error: %s", esp_err_to_name(ret));
        return;
    }
    fp = fopen("/spiflash/ftest.txt", "wb");
    if (fp) {
        fputs("external spi flash fat filesystem test, LINE 1\n", fp);
        fputs("external spi flash fat filesystem test, LINE 2\n", fp);
        fputs("external spi flash fat filesystem test, LINE 3\n", fp);
        fclose(fp);
    } else {
        ESP_LOGI(TAG, "open external flash write fat file error(write)\n");
        return;
    }
    ESP_LOGI(TAG, "external spi flash write finished");
    fp = fopen("/spiflash/ftest.txt", "rb");
    if(fp) {
        while(fgets(buf, 100, fp)) {
            printf(buf);
        }
        fclose(fp);
        printf("\n");
    }else  {
         ESP_LOGI(TAG, "open external flash write fat file error(read)\n");
         return;
    }
    
}
void unregister_external_partition() {
    if (nvs_part != NULL)
        esp_partition_deregister_external(nvs_part);
    if (fat_part != NULL)
        esp_partition_deregister_external(fat_part);
    if (spiffs_part != NULL)
        esp_partition_deregister_external(spiffs_part);
}
 
void external_spiffs_read_test() {
    esp_err_t ret;
    size_t t,u;
    FILE *fp;
    char buf[100];
   
    
    // test external flash spiffs
    esp_vfs_spiffs_conf_t extspiffsConf = {
        .base_path="/spiffs",
        .partition_label = extern_flash_spiffs, 
        .max_files=5,
        .format_if_mount_failed=true
    };

    ESP_LOGI(TAG, "external flash spiffs read test");
    ret = esp_vfs_spiffs_register(&extspiffsConf);
    if (ret != ESP_OK) {
        ESP_LOGI(TAG,"external flash spiffs register error: %s\n",esp_err_to_name(ret));
        return;
    }
    esp_spiffs_info(extern_flash_spiffs, &t, &u);
    ESP_LOGI(TAG, "external flash spiffs size -- Total:%d, used:%d\n", t,u);

    fp = fopen("/spiffs/mtest.txt", "r");
    while(fgets(buf, 100, fp)) {
        printf("read from external flash spiffs again ----- %s",buf);
    }
    fclose(fp);
    printf("\n");
   esp_vfs_spiffs_unregister(extern_flash_spiffs);
    
}
void app_main(void)
{
    flash_chip = malloc(sizeof(esp_flash_t));
    if (!external_flash_init()) {
        ESP_LOGI(TAG, "external flash init error");
        return;
    }
//esp_flash_erase_chip(flash_chip);
    if (!register_external_partition()) {
        ESP_LOGI(TAG, "register external flash partition error");
        return;
    }

    nvs_read_write_test();
    spiffs_read_write_test();
    fat_read_write_test();
    external_spiffs_read_test();

    unregister_external_partition();
    

}