Skip to content

Latest commit

 

History

History
239 lines (187 loc) · 9.86 KB

README.en.md

File metadata and controls

239 lines (187 loc) · 9.86 KB
Raw

M5Stack_FlipBookSD

日本語

Demo

demo_hq.mp4

SINTEL (Trailer) Creative Commons Attribution 3.0
© copyright Blender Foundation | www.sintel.org
The data format is modified for playback.

Overview

This application playback streams movie files that converted to the dedicated format gmv from SD.
It uses multi-cores to perform rendering with DMA and audio playback.
The old format gcf + wav is no longer playable since 0.1.1. Please regenerate it in gmv format or convert it using the gcf + wav => gmv conversion script.

Target devices

It must be able to run the libraries it depends on and have an SD card.

  • M5Stack Basic 2.6 or later
  • M5Stack Gray
  • M5Stack Core2
  • M5Stack CoreS3

Required libraries

Supported libraries (not required)

Included

  • TJpgDec Version modified by Lovyan03 - san

Build type (PlatformIO)

For Basic,Gray,Core2

Env Description
release Basic Settings
release_DisplayModule Support DisplayModule
release_SdUpdater Support SD-Updater
release_SdUpdater_DisplayModule Support DisplayModule and SD-Updater

For CoreS3

Env Description
S3_release Basic Settings
S3_release_DisplayModule Support DisplayModule

Sample data for playback

Download sample_0_1_1.zip, unzip it and copy to /gmv on your SD card.

How to make data

Required tools

(See also each webpage for installation)

Procedure

Making data on terminal.
Movie data can be in any format that can be processed by FFmpeg.

  1. Copy movie data to an arbitrarily created directory.
  2. Copy conv.sh and gmv.py to the same directory.
  3. Execute the shell script as follows
    bash conv.sh movie_file_name frame_rate [ jpeg_maxumum_size (Default if not specified is 7168) ]
Argument Required? Description
movie_file_path YES Source movie
frame_rate YES Output frame rate (1.0 - 30.0)
Integer or decimal numbers can be specified
jpeg_maximum_size NO Maximum file size of one image to output (1024 - 10240)
Larger sizes preserve quality but are more likely to cause processing delays (see "Known Issues").
  1. The files that named "movie_file_name.gmv" output to same directory.
  2. Copy the above files to /gmv on the SD card.

e.g.)

mkdir foo
cp bar.mp4 foo
cp script/conv.sh foo
cp script/gmv.py foo
cd foo
bash conv.sh bar.mp4 29.97
cp bar.gmv your_sd_card_path/gmv

Processes performed by shell scripts

  • Output JPEG images from movie at the specified frame rate.
    Create an output directory of . /jpg+PID as the output directory. This allows multiple terminals to convert in parallel.
  • If the size of the output JPEG file exceeds the specified size, reconvert it to fit.
  • Output audio data from the movie and normalize it out.
  • gmv.py creates a dedicated file containing images and audio.

Parameters of FFmpeg

ffmpeg -i $1 -r $2 -vf scale=320:-1,dejudder -qmin 1 -q 1 jpg$$/%06d.jpg

You can change the output quality, filters, etc. to your liking. The best parameters depend on the source movie, so please refer to FFmpeg's information.

Data restrictions

  • Movie formats that can be converted
    Formats that FFMpeg cannot handle are not supported.

  • wav data quality (8KHz unsigned 8bit mono)
    The quality of the audio data is lowered to reduce the processing load.
    Scripts can be edited to improve quality, but processing delays may occur due to processing load. (See Known Issues)

  • Image size and frame rate
    When converting a movie to JPEG, the width is 320px and the height is a value that maintains the aspect ratio.
    Currently, 320 x 240 can be played back at about 24 FPS, and 320 x 180 at about 30 FPS.
    To change the image size, edit the parameter for FFmpeg in conv.sh. (scale=)

  • Image size and output device size
    If the image size is narrower or wider than the output device size, it will be centered.

Movie data search

Searches for files in /gmv. If it does not exist, the old version /gcf is searched.
If both exist, only /gmv is searched.

Known issues

Audio is choppy or playback speed is slow.

This may be due to the processing not being completed in time within a frame.
In rare cases, it may take a long time to read from the SD card, which may cause some frames to not be processed in time.
There may be a problem with the compatibility or formatting of the SD card.

greiman/SdFat#96 (comment)

OS utilities should not be used for formatting SD cards. FAT16/FAT32 has lots of options for file system layout. In addition to the cluster size there are options for aligning file structures.
The SD Association has a standard layout for each size SD card. Cards are designed to optimize performance for the standard layout. For example, flash chip boundaries are aligned with file system structures.
My SdFormatter example produces the standard layout. On a PC use the SD Association Formatter.
You should not be getting errors due to the format. The correct format will only enhance performance, not reduce errors.
I rarely see the type errors you are having. Most users either have solid errors or no errors.
I have seen this type error when another SPI device interferes with the SD or when there are noisy or poor SPI signals.

Workaround by the data

  • Reduce playback frame rate
bash conv.sh movie.mp4 30 # 30 FPS
bash conv.sh movie.mp4 24 # Reduce to 24
  • Reduce JPEG file size
bash conv.sh movie.mp4 30      # 7168 as default
bash conv.sh movie.mp4 30 5120 # Reduce to 5120
  • Reduce image size
conv.sh
# ...
#ffmpeg -i $1 -r $2 -vf scale=320:-1,dejudder -qmin 1 -q 1 jpg$$/%06d.jpg  # 320 x n pixel
 ffmpeg -i $1 -r $2 -vf scale=240:-1,dejudder -qmin 1 -q 1 jpg$$/%06d.jpg  # 240 x n pixel
# ...

How to operate

Menu

Button Description
Click A To previous file
Click B Playback current file
Click C To next file
Hold A Change playback method
Hold C Change playback method

During playback

Button(Basic,Gray,Core2) Touch(CoreS3) Description
Press A Press left 1/3 of the screen Decrease sound volume
Click B Click center 1/3 of the screen Stop payback and back to menu
Press C Press right 1/3 of the screen Increase sound volume

Conversion from old format (gcf + wav)

Pyhton script for conversion gcf_to_gmv.py and shell script for conversion of files in the current directory convert_gcf_to_gmv.sh for converting files in the current directory.

# gcf_dir has gcf + wav files
cp script/gcf_to_gmv.py gcf_dir
cp script/convert_gcf_to_gmv.sh gcf_dir
cd gcf_dir
bash convert_gcf_to_gmv.sh
cp *.gmv your_sd_card_path/gcf

Digression

Why combine all the JPEG files together?

Opening and seeking files on an SD card takes a fair amount of time.
When JPEG files are opened and displayed one at a time, the speed becomes a bottleneck.
Therefore, we have adopted the method of sequentially loading a large single file, which was effective for physical media (floppy disks, etc.) and optical media (CDs, etc.), to shorten the card access time.

Initially, we used unzipLIB to read from uncompressed ZIP files, but since the file is internally seek once, we developed an original file format that assumes sequential reading without seeking, and by using this format we were able to eliminate the seeking.

The read processing time per image has gone from tens of ms for each file opened to a few ms.

Digression of the digression

I was experimenting with unzipLIB because I wanted to handle ZIP files. I wanted to learn how to use it, and I wanted to play back a collection of image files like a flip book.
(In the end, I ended up not using unzipLIB (´･ω･`) )

Is this different from MotionJPEG?

According to the definition, this can be called MJPEG. However, MJPEG is not a unified file format, so this application is only a gmv format player.

Appendix

  • src/gob_jpg_sprite.hpp
  • src/gob_jpg_sprite.cpp
    This class is for JPEG output using multi-cores to LGFX_Sprite instead of screen.
    I made it when I was doing a lot of trial and error, but ended up not using it.
    Too good to waste, so I include it in the package.

Acknowledgments

Those who have been the forerunners in the flip-book program. Your activities have inspired us to create this program.

The author of TJpgDec

I borrowed and modified the logic for DMA drawing to the screen using TJpegDec.
He also gave me lots of technical advice.
And he is the author of M5Unified and M5GFX.