此例程是运行FOMO 轻量检测模型实现物体检测，Demo中已包含训练好的模型参数，无需再训练。
FOMO(Faster Objects, More Objects) 是由 Edgeimpulse 工程师提出的一种轻量级的目标检测模型，其主要特点是模型非常小，计算量也很小，表现出来就是速度非常快，精度也不错。FOMO 的优点是速度快，缺点是精度不高，但是在一些对精度要求不高的场景下，FOMO 是一个不错的选择。
FOMO介绍：FOMO 轻量检测模型 - Sipeed Wiki
\edgelab-example-esp32\examples\fomo_detection_demo
编程语言C++。
app_main.cpp //主文件，加载模型，获取图像，算法检测，打印结果
fomo_mobilenetv2_model_data.cpp //模型参数文件
fomo_mobilenetv2_model_data.h //模型参数头文件
extern "C" void app_main(void) {
using namespace edgelab; //使用edgelab。
Device* device = Device::get_device(); //创建一个Device对象，Device类在\edgelab-example-esp32\components\sscma-micro\porting\el_device.h文件中定义。
Display* display = device->get_display(); //定义一个显示器变量并赋值，但XIAO ESP32S3套件没有显示屏，需要去掉。
Camera* camera = device->get_camera(); //定义一个相机变量并赋值。Camera类在\edgelab-example-esp32\components\sscma-micro\porting\el_camera.h文件中定义。
display->init(); //显示屏初始化，但XIAO ESP32S3套件没有显示屏，需要去掉。
camera->init(240, 240); //摄像头初始化。
auto* engine = new EngineTFLite(); //创建TFLite模型引擎。
auto* tensor_arena = heap_caps_malloc(kTensorArenaSize, MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT); //从SPIRAM中分配kTensorArenaSize (1024 * 1024)个8位宽的内存。
engine->init(tensor_arena, kTensorArenaSize); //初始化引擎。
engine->load_model(g_fomo_mobilenetv2_model_data, g_fomo_mobilenetv2_model_data_len); //引擎加载模型参数。
auto* algorithm = new AlgorithmFOMO(engine); //创建一个FOMO模型算法对象。
while (true) {
el_img_t img; //定义图像变量。
camera->start_stream(); //启动摄像头拍摄视频流。
camera->get_frame(&img); //获取一帧图像数据。
algorithm->run(&img); //运行算法检测图像中是否有物体。
uint32_t preprocess_time = algorithm->get_preprocess_time();
uint32_t run_time = algorithm->get_run_time();
uint32_t postprocess_time = algorithm->get_postprocess_time();
uint8_t i = 0u;
for (const auto& box : algorithm->get_results()) {
el_printf("\tbox -> cx_cy_w_h: [%d, %d, %d, %d] t: [%d] s: [%d]\n",
box.x,
box.y,
box.w,
box.h,
box.target,
box.score);
int16_t y = box.y - box.h / 2;
int16_t x = box.x - box.w / 2;
el_draw_rect(&img, x, y, box.w, box.h, color[++i % 5], 4); //在图片上加入标识框。
}
el_printf("preprocess: %d, run: %d, postprocess: %d\n", preprocess_time, run_time, postprocess_time);
display->show(&img); //显示屏显示图像，但XIAO ESP32S3套件没有显示屏，需要去掉。
camera->stop_stream(); //停止相机拍摄视频流。
}
delete algorithm;
delete engine;
}先去掉app_main()中display相关的代码。
在Device* device = Device::get_device();之后加入以下两行代码：
device->init();
printf("device_name:%s\n", device->get_device_name() );这是Demo的一个bug，如果不加入，程序运行会出现内存错误，ESP32S3会重启。
执行以下指令：
cd fomo_detection_demo
idf.py set-target esp32s3
idf.py build编译成功后日志如下：
Creating esp32s3 image...
Merged 2 ELF sections
Successfully created esp32s3 image.
Generated F:/esp/esp-idf/projects/edgelab-example-esp32/examples/fomo_detection_demo/build/fomo_detection_demo.bin
[1263/1263] cmd.exe /C "cd /D F:\esp\esp-idf\projects\edge...xamples/fomo_detection_demo/build/fomo_detection_demo.bin"
fomo_detection_demo.bin binary size 0x1327a0 bytes. Smallest app partition is 0x3bc000 bytes. 0x289860 bytes (68%) free.
Project build complete. To flash, run:
idf.py flash
or
idf.py -p PORT flash
or
python -m esptool --chip esp32s3 -b 460800 --before default_reset --after hard_reset --no-stub write_flash --flash_mode dio --flash_size 8MB --flash_freq 80m 0x0 build\bootloader\bootloader.bin 0x8000 build\partition_table\partition-table.bin 0x10000 build\fomo_detection_demo.bin
or from the "F:\esp\esp-idf\projects\edgelab-example-esp32\examples\fomo_detection_demo\build" directory
python -m esptool --chip esp32s3 -b 460800 --before default_reset --after hard_reset --no-stub write_flash "@flash_args"执行idf.py flash命令，开发板会自动进入boot模式。
使用idf.py flash或idf.py -p PORT flash指令进行烧录。第一个命令没有指定COM编号，如果电脑当前只有一个COM可以用此指令。如果电脑接了多个COM，需要用第二条指令，其中PORT为COMx，如idf.py -p COM5 flash。
执行idf.py monitor指令开启运行监视功能。
运行日志：
I (303) esp_image: segment 3: paddr=00129238 vaddr=3fca19I (738) esp_psram: SPI SRAM memory test OK
I (747) cpu_start: Pro cpu start user code
I (747) cpu_start: cpu freq: 240000000 Hz
I (747) cpu_start: Application information:
I (747) cpu_start: Project name: fomo_detection_demo
I (747) cpu_start: App version: 040a7b4-dirty
I (748) cpu_start: Compile time: Dec 17 2023 00:42:49
I (748) cpu_start: ELF file SHA256: 08b96bf32...
I (748) cpu_start: ESP-IDF: v5.3-dev-741-g03414a1550-dirty
I (748) cpu_start: Min chip rev: v0.0
I (749) cpu_start: Max chip rev: v0.99
I (749) cpu_start: Chip rev: v0.1
I (749) heap_init: Initializing. RAM available for dynamic allocation:
I (749) heap_init: At 3FCA7A98 len 00041C78 (263 KiB): RAM
I (749) heap_init: At 3FCE9710 len 00005724 (21 KiB): RAM
I (750) heap_init: At 600FE010 len 00001FD8 (7 KiB): RTCRAM
I (750) esp_psram: Adding pool of 8192K of PSRAM memory to heap allocator
I (751) spi_flash: detected chip: gd
I (751) spi_flash: flash io: qio
W (751) i2c: This driver is an old driver, please migrate your application code to adapt `driver/i2c_master.h`
I (752) sleep: Configure to isolate all GPIO pins in sleep state
I (752) sleep: Enable automatic switching of GPIO sleep configuration
I (753) main_task: Started on CPU0
I (763) esp_psram: Reserving pool of 32K of internal memory for DMA/internal allocations
I (763) main_task: Calling app_main()
device_name:Seeed Studio XIAO (ESP32-S3)
I (763) s3 ll_cam: DMA Channel=4
I (763) cam_hal: cam init ok
I (763) sccb: pin_sda 40 pin_scl 39
I (763) sccb: sccb_i2c_port=1
I (773) camera: Detected camera at address=0x30
I (773) camera: Detected OV2640 camera
I (773) camera: Camera PID=0x26 VER=0x42 MIDL=0x7f MIDH=0xa2
I (843) s3 ll_cam: node_size: 3840, nodes_per_line: 1, lines_per_node: 8
I (843) s3 ll_cam: dma_half_buffer_min: 3840, dma_half_buffer: 11520, lines_per_half_buffer: 24, dma_buffer_size: 23040
I (843) cam_hal: buffer_size: 23040, half_buffer_size: 11520, node_buffer_size: 3840, node_cnt: 6, total_cnt: 10
I (843) cam_hal: Allocating 115200 Byte frame buffer in PSRAM
I (843) cam_hal: cam config ok
I (843) ov2640: Set PLL: clk_2x: 1, clk_div: 3, pclk_auto: 1, pclk_div: 8
preprocess: 4, run: 51, postprocess: 0
preprocess: 4, run: 51, postprocess: 0
preprocess: 4, run: 51, postprocess: 0
preprocess: 4, run: 51, postprocess: 0
preprocess: 4, run: 51, postprocess: 0
preprocess: 3, run: 51, postprocess: 1
preprocess: 3, run: 51, postprocess: 1
preprocess: 3, run: 51, postprocess: 0
preprocess: 3, run: 51, postprocess: 0
preprocess: 3, run: 51, postprocess: 0
preprocess: 3, run: 51, postprocess: 0
preprocess: 3, run: 51, postprocess: 0
preprocess: 4, run: 51, postprocess: 0
preprocess: 4, run: 51, postprocess: 0
preprocess: 4, run: 51, postprocess: 0
preprocess: 4, run: 51, postprocess: 0
preprocess: 4, run: 51, postprocess: 0
preprocess: 4, run: 51, postprocess: 0
preprocess: 4, run: 51, postprocess: 0
preprocess: 4, run: 51, postprocess: 0
preprocess: 4, run: 51, postprocess: 0
preprocess: 4, run: 51, postprocess: 0
preprocess: 3, run: 51, postprocess: 1
box -> cx_cy_w_h: [90, 190, 20, 20] t: [1] s: [85]日志""打印"box -> cx_cy_w_h:"表示检测到了物体，后面的数值是物体的区域。
Waiting for the device to reconnect.
I (303) esp_image: segment 3: paddr=00129220 vaddr=3fca19I (738) esp_psram: SPI SRAM memory test OK
I (747) cpu_start: Pro cpu start user code
I (747) cpu_start: cpu freq: 240000000 Hz
I (747) cpu_start: Application information:
I (747) cpu_start: Project name: fomo_detection_demo
I (747) cpu_start: App version: 040a7b4-dirty
I (748) cpu_start: Compile time: Dec 17 2023 00:42:49
I (748) cpu_start: ELF file SHA256: 95d9277c7...
I (748) cpu_start: ESP-IDF: v5.3-dev-741-g03414a1550-dirty
I (748) cpu_start: Min chip rev: v0.0
I (749) cpu_start: Max chip rev: v0.99
I (749) cpu_start: Chip rev: v0.1
I (749) heap_init: Initializing. RAM available for dynamic allocation:
I (749) heap_init: At 3FCA7A98 len 00041C78 (263 KiB): RAM
I (749) heap_init: At 3FCE9710 len 00005724 (21 KiB): RAM
I (750) heap_init: At 600FE010 len 00001FD8 (7 KiB): RTCRAM
I (750) esp_psram: Adding pool of 8192K of PSRAM memory to heap allocator
I (751) spi_flash: detected chip: gd
I (751) spi_flash: flash io: qio
W (751) i2c: This driver is an old driver, please migrate your application code to adapt `driver/i2c_master.h`
I (751) sleep: Configure to isolate all GPIO pins in sleep state
I (752) sleep: Enable automatic switching of GPIO sleep configuration
I (752) main_task: Started on CPU0
I (762) esp_psram: Reserving pool of 32K of internal memory for DMA/internal allocations
I (762) main_task: Calling app_main()
Guru Meditation Error: Core 0 panic'ed (LoadProhibited). Exception was unhandled.
Core 0 register dump:
PC : 0x42009618 PS : 0x00060c30 A0 : 0x820d815e A1 : 0x3fcab5c0
0x42009618: app_main at F:/esp/esp-idf/projects/edgelab-example-esp32/examples/fomo_detection_demo/main/app_main.cpp:27
A2 : 0x00000003 A3 : 0x3c0e405c A4 : 0x00000000 A5 : 0x000002fa
A6 : 0x3c0e405c A7 : 0x3fcab620 A8 : 0x82009616 A9 : 0x3fcab5a0
A10 : 0x3fca3d80 A11 : 0x3c0e405c A12 : 0x3c0e4128 A13 : 0x3fcab600
A14 : 0x3fcab5e0 A15 : 0x3fca3d80 SAR : 0x0000000c EXCCAUSE: 0x0000001c
EXCVADDR: 0x00000000 LBEG : 0x400570e8 LEND : 0x400570f3 LCOUNT : 0x00000000
0x400570e8: memset in ROM
0x400570f3: memset in ROM从以上日志可以看出执行camera->init(240, 240);代码让ESP32重启了。
warning: unknown kconfig symbol 'CAMERA_MODULE_ESP_S3_EYE' assigned to 'y' in F:/esp/esp-idf/projects/edgelab-example-esp32/examples/fomo_detection_demo/sdkconfig.defaults.esp32s3解决方法：参看3.2节。
本文章为转载内容，我们尊重原作者对文章享有的著作权。如有内容错误或侵权问题，欢迎原作者联系我们进行内容更正或删除文章。
