Skip to content

disheng222/H5Z-SZ

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

28 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

#H5Z-SZ #the filter for integrating SZ into HDF5

#Quick start

Step 1: Download SZ package from : https://github.com/disheng222/SZ

Step 2: Install SZ by using "./configure --prefix=[INSTALL_PATH];make;make install"

  • e.g., ./configure --prefix=/home/sdi/Install/sz-1.4.10-beta-install

Step 3: You can go to example/ directory to test whether the installation of SZ is fine by the following commend: ./test.sh

Step 4: Download the H5Z_SZ_filter (In fact, H5Z_SZ_filter has been integrated in SZ package: see the hdf5_filter directory)

Step 5: Open Makefile and modify the SZ path and HDF5 path based on your local setting: SZPATH = /home/sdi/Install/sz-1.4.10-beta-install HDF5PATH = /home/sdi/Install/hdf5-1.10.1-install

Step 6: Execute "make;make install"

Step 7: Set HDF5_PLUGIN_PATH

  • e.g., export HDF5_PLUGIN_PATH=$SZPATH/lib

Step 8: Set the environment variable LD_LIBRARY_PATH as follows: export LD_LIBRARY_PATH=$HDF5_HOME/lib:$SZ_HOME/lib:$LD_LIBRARY_PATH

Now, you have finished the installation of H5Z-SZ filter.

#There are two use-cases:

Testing method 1 (with library):

Step 1: Then, go to the ./test/ directory, and edit its Makefile by setting SZPATH and HDF5PATH similarly.

Step 2: make (Note: you need to use $(MPICC) instead of $(CC) to compile the package, if you are using parallel version of HDF5)

Step 3: Run test_compress.sh and test_decompress.sh to do the test (testing data is in the directory testdata/ of the package).

Testing method 2 (with plugin):

Step 1: Put the sz.config configuration in the current directory. (Please see README in SZ to understand the configuration sz.config)

Step 2 (Compression): h5repack.sh [input_hdf5_file] [compressed_hdf5_file] or h5repack -f UD=32017,0 [input_hdf5_file] [compressed_hdf5_file]

Step 3 (Decompression): Read the compressed HDF5 file: h5dump [compressed_hdf5_file] > data.txt


##Trouble shooting

1. When using "make" command to compile test/, I ran into the following error: szToHDF5.c:15:20: fatal error: H5Z_SZ.h: No such file or directory
Answer: You probably forgot to execute "make install" after you use "make" to compile the H5Z-SZ package. Run "make install" will solve the problem.

Contact: 
sdi1@anl.gov

About

hdf5 filter based on SZ compressor

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published