Silo

July 21, 2014
Announcing the release of version 4.10 of the Silo Library

Obtaining Silo

• Obtain BSD Licensed source code at https://wci.llnl.gov/sites/wci/files/2021-01/silo-4.10-bsd.tgz
• Obtain BSD Licensed small test source code at https://wci.llnl.gov/sites/wci/files/2021-01/silo-4.10-bsd-smalltest.tgz
• Obtain Legacy Licensed source code at https://wci.llnl.gov/sites/wci/files/2021-01/silo-4.10.tgz
• Obtain documentation at https://wci.llnl.gov/content/assets/docs/simulation/computer-codes/silo/LLNL-SM-654357.pdf

Organization of information in this announcement

1. Important information regarding this release.
2. Bugs fixed in this release.
3. New features in this release.
4. Removal of Legacy Datatype'd pointers.
5. Const qualification and multi-level arrays.

1. Important information regarding this release.

• Until the release tarballs are permanently available at the URL's mentioned above (probably next week sometime), they will be temporarily available at...
  • ftp://ftp.llnl.gov/outgoing/silo-4.10.tar.gz
  • ftp://ftp.llnl.gov/outgoing/silo-4.10-bsd.tar.gz
  • ftp://ftp.llnl.gov/outgoing/silo-4.10-bsd-smalltest.tar.gz
• Version 4.10 is NOT link-time compatible with any previous version. You will need to re-compile and re-link your applications.
• In some cases, passing a 'make check' requires adding 'LIBS=-lstdc++' to the make command line (e.g. 'make check LIBS=-lstdc++'.
• The rocket test will fail during 'make check' on OSX. We've yet to learn how to handle dlopen and shared libs there. For similar reasons, Silo's python module has yet to be tested on a OSX.

2. Bugs fixed in this release

• Fixed DBMakeNamescheme to automagically handle nameschemes that included external array references. In previous release, the caller was required to scan the namescheme for any extenal array references and read them from the file in a separate step. Now, this work is handled automatically by DBMakeNamescheme.
• Fixed erroneous display of empty material and quadvar objects in browser.
• Fixed incorrect documentation for 1-origin indexing of matspecies object.
• Fixed missing documentation for return value for Fortran interface dbcreate/dbopen.
• Fixed documentation for PMPIO_RankInGroup to clearly indcate caller can obtain rank of any processor in a group.
• Changed name of CheckVersion macro to SiloCheckVersion to avoid possible name collision with similar symbols from other packages.
• Fixed documentation for namescheme construction with external array references.
• Fixed erroneous error message regarding missing HDF5 driver when opening no-writeable file in browser.
• Improved manner in which empty objects are created relieving caller of several tedious constraints in setting argument lists just right to create empty objects.
• Fixed Silo's Fortran header, silo.inc, so that it is compatible with free-format 9x. Super thanks to Bob Apthorpe for this fix!
• Fixed browser's display of some objects containing variable length lists of arrays as well as failing to include such arrays in diffs produced by silodiff.
• Fixed handling of complex indexing expressions in external array indices in nameschemes.
• Fixed handling of ?:: operator in nameschemes to not attempt to evaluate whichever path is not taken.
• Fixed documentation of species example in user's manual to show correct percentages for O-1 steel.
• Fixed a number of compilation warnings from OS X (clang) compiler.
• Fixed misleading ***ERROR*** message from browser when exiting and command history is not able to be saved.
• Fixed misleading display of repr_block_idx in silex.
• Fixed documentation regarding node-ordering for DB_ZONETYPE_POLYHEDRON zones.
• Improved ducmentation regarding polyhedral zonelists for 2D meshes.
• Fixed error in documentation regarding DBPutCompoundarray for Fortran callers.
• Added Fortran equiv. of DBPutQuadvar to documentation.
• Removed silly make dependencey of silock on browser in order to build silock.
• Fixed silex display of 'topo_dim' member for ucd meshes.
• Fixed diff'ing of curve objects in browser when reading HDF5 data.
• Added missing documentation for Fortran dbaddcaopt.
• Fixed file and library version comparison (GE) functions and macros.
• Fixed a leak in HZIP compression library.
• Fixed some leaks in browser tool using 'diff' operator.
• Fixed issues writing and reading Silo standard objects via custom object interface (DBMakeObject/DBWriteObject).
• Fixed compile time knowledge of Silo version digits such that blank digits (e.g. no pre or patch digit) are zeros at compile time.

3. New features in this release

• The entire Silo interface has been const-qualified where appropriate.
• It is now possible for PDBLite API to be exported publicly as part of a Silo installation. Use the --enable-install-lite-headers configuration option to enable this. This frees applications which have been using PDBLite interface for some time to have to maintain their own CPP mapping macros.
• Improved DBoptlist processing so that caller needn't worry about exceeding maximum number of options specified in DBMakeOptlist call. Now, Silo will silently reallocate to accomodate going beyond maximum option count specified.
• Improved DBobject processing similarly so that caller needn't worry about exceeding maximum number of object components specified in DBMakeObject. Now, Silo will silently reallocate to accomodate going beyond the maximum component count specified.
• Added support to all variable writing routines for missing data using the DBOPT_MISSING_VALUE optlist option. Support for this Silo feature has also been added to VisIt and will be available in VisIt 2.8.
• Added support for data producer to specify ghosting of meshes by writing extra mesh (or generic var) data and using DBOPT_GHOST_NODE_LABELS and/or DBOPT_GHOST_ZONE_LABELS optlist options. This is as an alternative to hi/lo indexing method previously used for ghost zone enumeration. Support for this Silo feature is not yet available in VisIt but will be added after VisIt 2.8 is released.
• Added support for data producer to specify alternative node or zone numberings using DBOPT_ALT_ZONENUM_VARS and DBOPT_ALT_NODENUM_VARS optlist options. Support for this Silo feature is not yet available in VisIt but will be added after VisIt 2.8 is released.
• Changed browser's default setting to open files for read-only instead of read-write.
• Added ability to open a Silo "file" from a buffer of bytes (e.g. the File-In-Core). Note, this is different from the "core" file driver where a file is first read entirely into core before it is opened because it does not require or start from a file on disk.
• Added a convenience method, DBOpenByBcast(), for parallel applications to open and process a root or master file. This method will open the file on a single processor and then broadcast its entire contents to all processors all of which turn around and open the resulting buffer of bytes as a Silo file. Note that this requires HDF5-1.8.10 or later.
• Added support for Silo curves in polar coordinates.
• Improved time and space performance of DBGetMultiXXX calls' processing long lists of catenated strings at large block counts. Reads of legacy files containing these objects should be faster. Writes of such objects should be faster too.
• DB[Set|Get]DataReadMask using long type was replaced with DB[Set|Get]DataReadMask2 using long long type.
• DBVersionDigits and DBFileVersionDigits were added to return integer major, minor, patch and pre-release version digits.
• DBVersionGEFileVersion was added to compare library version with file version.
• JSON support was added to Silo using the json-c library. This interface permits Silo applications to manipulate their Silo objects as JSON objects. The implementation includes extensions to json-c for efficient handling of problem-sized array data in Silo objects. This interface should be considered experimental and is likely to change, perhaps in dramatic ways, in future releases.
• Improved display of character array data in silex.
• netCDF driver was enabled on Windows.
• By default, Silo no longer configures the silex browser. You have to explicitly enable the silex browser using the --enable-silex configuration switch (Thanks to Alex Henrie).
• mkfort tool which produces fortran include files was enhanced to properly format both F77 and F9x header files (Thanks to Bob Apthorpe).
• There are now two separate Fortran include files; one for F77 and one for F9x.
• Improved silodiff tool's ability to process directories. It previously required files in both directories to have the same names. Now, it simply diffs files in two directories in reverse order of time of last modification.
• Enhanced both PDB and HDF5 drivers to support customization of standard objects; that is the ability to create an object via DBMakeObject/DBWriteObject that would have otherwise been created via a DBPutXxxx (e.g. DBPutQuadvar) call, such that callers of DBGetXxxx will see the object as they ordinarily would. This permits Silo clients to create user-defined customizations to Silo's standard objects.

4. Removal of Legacy Datatype'd pointers.

In a Nutshell...

This applies mainly to very old Silo applications prior to version 4.6. Explicitly casting to float** two-level arrays in calls to some of the older Silo functions (e.g. DBPutUcd{mesh|var} or DBPutQuad{mesh|var}) is no longer required and will now result in compiler warnings for C callers and compiler errors for C++ callers.

The details...

In the early years of Silo's development, several functions designed to accept two-level arrays of arbitrary type nonetheless defined the function's formal arguments as float** instead of void**. These were changed to void** years ago. However, the configuration option, --enable-legacy-datatyped-pointers, was provided to maintain backward compatability with this old interface.

As part of the work to const qualify the Silo interface, continuing to maintain this backward compatability was becoming too complex. The configuration option --enable-legacy-datatyped-pointers has been eliminated along with the convenience macros DB_DTPTRx (for x=1, 2 and 3).

If you have been in the habit of using the --enable-legacy-datatyped-pointers configuration option and are wondering what will happen when you try to use Silo 4.10 in your application, the most likely biggest issue you will face is to remove cases where have explicitly cast certain array arguments in Silo calls to float**.

5. Const qualification and multi-level arrays.

In a Nutshell...

• Wherever Silo can accept a two-level array of arbitrary type (e.g. float**, int**, double**), the function's formal parameter is defined to be a void const * instead of a void const * const *. It is nonetheless still treated as a void const * const * in all other respects.
• Wherever Silo accepts a constant array of constant strings (char const * const *), C callers will be required to explicitly cast char** and char const ** arguments to avoid compiler warnings.

The details...

Please refer to this discussion for a detailed explanation.

March 19, 2013

Obtaining Silo

• Obtain BSD Licensed source code at https://wci.llnl.gov/sites/wci/files/2021-01/silo-4.9.1-bsd.tgz
• Obtain BSD Licensed small test source code at https://wci.llnl.gov/sites/wci/files/2021-01/silo-4.9.1-bsd-smalltest.tgz
• Obtain Legacy Licensed source code at https://wci.llnl.gov/sites/wci/files/2021-01/silo-4.9.1.tgz
• Obtain documentation at https://wci.llnl.gov/content/assets/docs/simulation/computer-codes/silo/LLNL-SM-453191.pdf

Organization of information in this announcement

1. Important information regarding this release.
2. Bugs fixed in this release.

1. Important information regarding this release.

• Until the release tarballs are permanently available at the URL's mentioned above (probably next week sometime), they will be temporarily available at...
  • ftp://ftp.llnl.gov/outgoing/silo-4.9.1.tar.gz
  • ftp://ftp.llnl.gov/outgoing/silo-4.9.1-bsd.tar.gz
  • ftp://ftp.llnl.gov/outgoing/silo-4.9.1-bsd-smalltest.tar.gz
• Version 4.9.1 is link-time compatible with 4.9.
• The VisIt plugin in VisIt 2.6.2 to be released shortly, includes bugfixes related to nameschemes.
• In some cases, passing a 'make check' requires adding 'LIBS=-lstdc++' to the make command line (e.g., 'make check LIBS=-lstdc++'.
• The rocket test will fail during 'make check' on OSX. I've yet to learn how to handle dlopen and shared libs there. For similar reasons, Silo's python module has yet to be tested on a OSX.

2. Bugs fixed in this release

• Relative paths to external array references in nameschemes were fixed.
• A bug where poorly defined namescheme strings could cause segmentation violations was fixed.
• Argument checking for empty objects was relaxed significantly to make it easier for existing clients to push them.

January 24, 2013

Obtaining Silo

• Obtain BSD Licensed source code at https://wci.llnl.gov/sites/wci/files/2021-01/silo-4.9-bsd.tgz
• Obtain BSD Licensed small test source code at https://wci.llnl.gov/sites/wci/files/2021-01/silo-4.9-bsd-smalltest.tgz
• Obtain Legacy Licensed source code at https://wci.llnl.gov/sites/wci/files/2021-01/silo-4.9.tgz
• Obtain documentation at https://wci.llnl.gov/content/assets/docs/simulation/computer-codes/silo/LLNL-SM-453191.pdf

Organization of information in this announcement

1. Important informagion regarding this release.
2. Bugs fixed in this release.
3. Robustness features in this release.
4. New features in this release.
5. Documentation of new features.

1. Important information regarding this release.

• Until the release tarballs are permenantly available at the URL's mentioned above (probably next week sometime), they will be temporarily available at...
  • ftp://ftp.llnl.gov/outgoing/silo-4.9.tar.gz
  • ftp://ftp.llnl.gov/outgoing/silo-4.9-bsd.tar.gz
  • ftp://ftp.llnl.gov/outgoing/silo-4.9-bsd-smalltest.tar.gz
• Version 4.9 is not link-time compatible with any previous version. You will need to re-compile and re-link your applications.
• Unfortunately, there was no time to update the documentation prior to this release. Please refer to section 5 of these release notes for key documentation information regarding PDB Lite, Namescheme and Empty Object support.
• The VisIt plugin in VisIt 2.6.1 to be released shortly, includes enhancements for new namescheme constructor and empty objects.
• In some cases, passing a 'make check' requires adding 'LIBS=-lstdc++' to the make command line (e.g., 'make check LIBS=-lstdc++'.
• The rocket test will fail during 'make check' on OSX. I've yet to learn how to handle dlopen and shared libs there. For similar reasons, Silo's python module has yet to be tested on a OSX.

2. Bugs fixed in this release

• Added patch from Olivier Cessenat for DBSetCompression in Fortran API. Thanks Olivier.
• Fixed minor bug with guihide info for defvars.
• Fixed some leaks in DBFreeXXX methods.
• Fixed order of shuffle filter used in HDF5 compression pipelines for floating point data.
• Fixed a leak in HDF5 resources that would cause out of memory conditions for very large number of file opens and closes (tens of thousands). Thanks to Matt O'Brien for test case.
• Made reltol_eps diff algorithm mutually exclusive to abs||rel diff algorithms.
• Added patches from Neil Hodge to Fortran intercace for DBADDREGION and DBPUTGRPLMAP for null maps and segment_fracs. Thanks Neil.
• Fixed invalid formatting of silo.inc file and fixed mkinc tool that produces it. Thanks to Olivier Cessenat.
• Qt autoconf logic was updated to most current. Typically, something like the following is required:

  --with-Qt-dir= --with-Qt-lib=QtGui

• Extensive test for variety of arbitrary polyhedra methods were added.
• A performance bug in DBSetDir()/DBGetDir() methods was fixed. Thanks to Brian McCandless for reporting the bug and providing the fix.
• Return of static local variable from DBGetName was replaced with a circular buffer.
• DBtoc data member names were changed to all obey same pluralization rules. If you have code that examines the members of the DBtoc structure, be sure to review silo.h file for corrected names.
• A fortran test for csg meshes was added, csgmesh_f.f. Thanks to Robert Salko.
• Added the ability to specify driver via CL arguments to Fortran tests.
• Some bourne shell based unit tests were adjusted to work on OSX.

3. Robustness features in this release

• The Klocwork static analyzer was applied to address a number of possible memory faults and leaks. Thanks to Bill Oliver for helping with this effort.
• Used gcc mudflap stack memory analyzer to address possible memory faults and leaks.
• The SAFECode stack memory analyzer was used to address several memory faults and leaks.
• Testing and tools functionality was substantially improved on Windows. Thanks to Kathleen Biagas for doing this. This included getting browser tool to build/run on Windows, get 64 bit Windows builds working, getting Fortran interface on Windows working and updating to Visual Studio 2010.
• The test multi_test.c was enhanced substantially to test readback of data it writes and compare it.
• The testxvers test was fixed to run properly and filter irrelevant diffs.
• A test was added to specifically test the PDB Lite interface and lite_pdb.h and lite_score.h header files.

4. New features in this release

• The python module was enhanced substantially to read Silo object structures as Python dictionaries as well as write Python dictionaries as Silo objects. See examples in the tests directory.
• PDB Lite was enhanced to address time and space performance issues for files with large numbers (hundreds of thousands) of indirections.
• Support for Visual Studio 2010 was added.
• New extensive example in the use of mrgtrees and subsetting (rocket.C) was added.
• DBMakeNamescheme was enhanced to automatically find and open any external arrays referenced in the namescheme.
• Empty objects are now supported. An empty object is a silo object but with no data; a mesh with no nodes or zones or a material object with no materials. This is part of enabling Silo to scale to exascale.

5. Documentation of new features

PDB Lite

• PDB Lite (which has always sort of been a hidden library within Silo) header files (lite_pdb.h and lite_score.h) can now be optionally generated and installed. Use the --enable-install-lite-headers configuration option to enable this feature.
• The PDB Lite header files, lite_pdb.h and lite_score.h can be used in place of pdb.h and score.h for existing PDB code provided the code uses only those features available in the PDB Lite version of the library.
• Options were added to lite_PD_open/lite_PD_create to control the size of the hash table used to manage PDB Lite's symbol table. An 's', 'm', 'l', 'x' appearing in the mode string to lite_PD_open/lite_PD_create will use a small (521), medium (5231), large (52121) or extra-large (521021) symbol table. An 'i' appearing in the mode string will force PDB Lite to ignore indirection objects when loading the symbol table. For files containing large numbers (hundreds of thousands) of indirection objects, such as might be seen for certain equation of state databases, a good setting for the mode string of "rxi" (r-read, x-extra large hash table, i-ignore putting indirection objects in the symbol table.).
• For examples in the Silo release tarball, see tests/testpdb.c and tests/pdbtst.c
• On Windows, the build is set up for using PDB Lite only. Perl is now required to generate the PDB Lite header files. Please ensure perl is in your path, or modify Visual Studio settings to add its path to the 'executable directories'.

Namescheme Construction

• DBMakeNamescheme() was enhanced to also automatically find and bind externally referenced arrays to the constructed and returned DBnamescheme object. The proper syntax to call DBMakeNamescheme() is exemplified in the code snipit below:

  DBfile *dbfile;
  DBmultimesh *mm = DBGetMultimesh(dbfile, "foo");
  DBnamesheme *ns = DBMakeNamescheme(mm->file_ns, 0, dbfile);
  for (i = 0; i < mm->nblock; i++)
      DBGetName(ns, i);
  

  Note: the additional "0, dbfile" arguments to DBMakeNamescheme(). The externally referenced arrays named in the namescheme string (first argument to DBMakeNamescheme), must be in the current working directory (cwd) of the dbfile that is passed as the 3rd argument. The "0, dbfile" arguments are optional and if you are sure you are dealing with a namescheme that does not utilize externally referenced arrays, you can choose not to pass them.
• For examples in the Silo release tarball, see tests/namescheme.c.

Empty Objects in place of "EMPTY" block name keyword

DBSetAllowEmptyObjects(1);
DBPutQuadmesh(dbfile, "empty_quadmesh", 0,0,0,0,DB_DOUBLE,DB_COLLINEAR,0);
DBPutQuadvar1(dbfile, "qv1", "empty_quadmesh", 0,0,0,0,0,DB_DOUBLE,DB_ZONECENT,0);

• The "EMPTY" block name keyword convention used to identify blocks in multi-block objects that are not present in the database cannot be made to work scalably.
• Silo was enhanced to allow writers to write empty objects to the file. An empty object is something like a mesh without any coordinate arrays and zero nodes and zones or a variable without any data arrays. Ordinarily, Silo would indicate that writing such an object is an error. A new method was added to Silo to turn off error checking that would ordinarily prevent empty objects from being written; DBSet/GetAllowEmptyObjects(). By default, Silo does not permit empty objects and a writer wishing to produce them must call DBSetAllowEmptyObjects(1) to allow them. Empty objects will be permitted to be written to files until the caller calls DBSetAllowEmptyObjects(0).
• Note: that although the caller may pass zero for most arguments, the caller must sill pass valid library values (but nonetheless irrelevant) for certain other arguments like datatype, centering, coordtype, etc.
• For examples in the Silo release tarball, see tests/empty.c.

DBOPT_MB_REPR_BLOCK_IDX option for namescheme'd multi-block objects with empty blocks

• For namescheme'd multi-block objects, there is no explicit indication of which blocks in the object are empty and which are not.
• Part of VisIt's process to open a Silo file requires reading metadata from a non-empty block of a multi-block object.
• In truth, the real problem in this process is the Silo library in that key metadata is sort of distributed between multi-block and individual block objects.
• However, correcting this issue involves a larger enhancement to Silo than practical for this release.
• The solution for multi-block objects with nameschemes is that the writer must indicate the index of some non-empty block (any non-empty block will do) in the multi-block object that can serve as representative. That is handled via the DBOPT_MB_REPR_BLOCK_IDX option on the multi-block object. See examples in Silo release tarball; test/multi_file.c

September 20, 2010

This is the first, official BSD Open Source Licensed release of the Silo Library.

• Obtain BSD Licensed source code at https://wci.llnl.gov/sites/wci/files/2021-01/silo-4.8-bsd.tgz
• Obtain Legacy Licensed source code at https://wci.llnl.gov/sites/wci/files/2021-01/silo-4.8.tgz
• Obtain documentation at https://wci.llnl.gov/content/assets/docs/simulation/computer-codes/silo/LLNL-SM-453191.pdf

Organization of information in this announcement

• Bugs fixed in this release.
• Performance-oriented features in this release.
• Other new features in this release.
• Important information regarding BSD vs. Legacy Licensed release tarballs.

1. Bugs fixed in this release

• A bug where the Force Single behavior of the HDF5 driver was resulting in all objects' arrays being converted to float but then neglecting to set the associated 'datatype' member to DB_FLOAT was corrected.
• A few cases where Silo was erroneously leaving HDF5 objects open in a file, thereby causing the file itself to be left open in HDF5 even after DBClose was called have been corrected.
• Some problems building Silo on windows with newer compilers and with various configurations have been corrected. Thanks to Kathleen Bonnell for these fixes as well as porting a number of code updates to Windows and for laborious testing on Windows.
• The silofile utility was re-written to NOT use magic file numbers as that proved to be unreliable. Now the tool is much more robust at identifying silo files as well as their underlying format (PDB or HDF5).
• A bug swapping the slash character in the filename part of multi-block object block names when moving files between windows and Linux was fixed. Thanks to Al Nichols for reporting the and the fix.
• Leaks of optional species names and colors were corrected.

2. Performance-oriented features in this release

• To take advantage of some of the performance improvements described here, HDF5 Version 1.8.4 or newer is required.
• A number of options for using various Virtual File Drivers (VFDs) that come pre-packaged with the HDF5 library were added to Silo. The interface to controlling these VFDs was enhanced. See documentation on DBRegisterFileOptionsSet() in the Silo 4.8 user's manual for more information. See Overview of Silo block-based VFD and Scaling Studies of Silo block-based VFD for some details.
• There is a new, Silo-specific VFD designed specifically for BlueGene (e.g., BG/P and BG/Q) systems called the Silo block-based VFD. Use of this VFD is essential for adequate performance on BG/P systems. Thanks to Mike Collette and Richard Hedges for performing scaling studies to help identify performance issues on BG/P. Read more about this VFD and its options as well as a number of other VFDs and their options in the Silo 4.8 User's manual.
• Silo can now optionally link with and use PDB Proper in PACT using the --with-pdb-proper=path-to-pact-libs and 'DB_PDBP' for the 'type' argument in DBCreate/DBOopen calls. Thanks to Marty Marinak for testing and Stewart Brown for guidance in implementing this feature. This feature may be particularly important to codes using PDB driver on BlueGene systems (e.g., BG/P) because the PDB Proper in PACT has undergone substantial performance tuning for those systems.
• A new I/O performance benchmarking tool, ioperf, was added to Silo tests. This tool can be useful in comparing performance of the Poor Man's Parallel I/O strategy with any I/O library including sec2, stdio, PDB, HDF5, Silo as well as any others a user might wish to write plugins to test.
• A new function, DBSortObjectsByOffset() was added that given a list of object names an application wishes to read from a Silo file, will return the order in which the named objects appear in the file. This is useful in optimizing read operations from the file by potentially minimizing the frequency/number of seeks as well as improving efficiency of read-ahead.
• New options for using nameschemes instead of explicit lists of names of blocks for multi-block objects were added. These options can profoundly reduce the total size of multi-block objects as well as the root files into which they are written as well as the I/O time to read and write them.
• A performance issue where HDF5 driver was constantly writing and rewriting cycle, time and dtime for every DBPutXXX operation was corrected.
• A new friendly names option was added such that HDF5 datasets are in fact written as their friendly named variant in the current working directory instead of writing them in the magic /.silo directory with the friendly names being links to them. This may be important for scalability as the number of objects in the /.silo directory can otherwise grow quite large causing possible performance issues for HDF5 to handle the related metadata.

3. Other new features in this release

• Note that Silo Version 4.8 is NOT link-time, backward compatible with any previously released version. You will need to re-compile all code that includes the Silo header file, silo.h.
• Silo is now officially released with a BSD Open source license. Thanks to Eric Brugger for reviewing the approximately 150,000 lines of code as well as the LLNL IM Team for completing the review and release process.
• There is a new, public, redmine bug tracker for Silo; https://visitbugs.ornl.gov. Thanks to Sean Ahern and Oak Ridge National Labs for hosting it and thanks to Cyrus Harrison for setting it up. Before users can submit requests there, you will be required to register and create an account. But, that is a one-time operation. If you are already registered for the VisIt tracker, then you can get the Silo tracker automatically.
• A significant reduction in compiler symbol table global namespace clutter has been eliminated from Silo reducing possibility of symbol collision when linking large applications.
• A new error reporting level was added, DB_ALL_AND_DRVR. This level of error reporting is the same as DB_ALL except that it also includes any additional error reporting from the underlying driver.
• Support for arrays of strings of variable length was added to a number of functions in the Fortran interface that accept arrays of strings. Thanks to Olivier Cessenat for this patch.
• DBPUTQV which accepts a multi-component quadvar was added to the Fortran interface. Thanks to Olivier Cessenat for this implementation.
• A restriction on the number of components of a multi-component ucdvar or quadvar was relaxed so that now, for example, DBPutUcdvar can write a variable with 100 sub-variables.
• DBnamescheme objects now support references to external arrays of string value.
• DBErrFunc was deprecated. Use DBErrFuncname now. Also, a new method, DBErrfunc was added to return the function pointer for the error function set in the last most-recent DBShowErrors() call.
• A new function, DBErrlvl, was added to return the error reporting level set in the last, most-recent DBShowErrors() call.
• The test suite was enhanced substantially to run test on a variety of HDF5 virtual file drivers and library settings.
• Functions for catenating an array of strings into a single, semi-colon separated string and the reverse operation were added to the public API. See DBStringArrayToStringList and DBStringListToStringArray. These functions used to be solely internal to Silo but have proven to be useful for Silo applications needing to read and write arbitrary collections of strings.
• A new function, DBSetUnknownDriverPriorities was added to control the behavior of the unknown (e.g., DB_UNKNOWN) driver when opening files of unspecified low-level format type. Previously, the Silo library had a hard-coded sequence of drivers it would try to open a file using DB_UNKNOWN driver type. Now, that sequence can be specified by the Silo application.

4. Important information regarding BSD vs Legacy Licenses

As of Version 4.8, July, 2010, the Silo source code is now available under the standard BSD Open Source License. See COPYRIGHT file for BSD notice.

The source code for two compression libraries which have been part of previous releases of the Silo library is not available under the terms of the BSD Open Source license. These are the Hzip and FPzip compression libraries.

For this reason, two different releases of the Silo source code are made available. One is the BSD Open Source release identified by the moniker 'bsd'in the distribution file names. For example, silo-4.8-bsd.tar.gz is the name of a BSD Open Source release distribution of Silo. The other is a non-Open Source release of Silo under the same licensing terms that all previous releases have been made. We call this other release a "Legacy" release. Legacy license terms are repeated below for convenience.

Copyright (c) 1994-2010, Lawrence Livermore National Security, LLC. All rights reserved. This work was produced at Lawrence Livermore National Laboratory under Contract No. DE-AC52-07NA27344 with the DOE.

DISCLAIMER

This work was prepared as an account of work sponsored by an agency of the United States Government. Neither the United States Government nor the University of California nor any of their employees, makes any warranty, express or implied, or assumes any liability or responsibility for the accuracy, completeness, or usefulness of any information, apparatus, product, or process disclosed, or represents that its use would not infringe privately-owned rights. Reference herein to any specific commercial products, process, or service by trade name, trademark, manufacturer or otherwise does not necessarily constitute or imply its endorsement, recommendation, or favoring by the United States Government or the University of California. The views and opinions of authors expressed herein do not necessarily state or reflect those of the United States Government or the University of California, and shall not be used for advertising or product endorsement purposes.

NOTIFICATION OF COMMERCIAL USE

Commercialization of this product is prohibited without notifying the Department of Energy (DOE) or Lawrence Livermore National Laboratory (LLNL).

The BSD Open Source release distributions do not contain the Hzip and FPzip compression libraries while the legacy release distributions do. This means any objects within a Silo file containing data that is compressed with Hzip or FPzip from a legacy distribution will not be readable in a BSD distribution. Since compression features must be explicitly turned on by a Silo application, this impact can be easily mitigating by ensuring that Hzip and FPzip compression are used only for objects within Silo files not intended to be read by BSD releases.

For more information on availability of Hzip and FPzip compression, contact the developer, Peter Lindstrom, lindstrom2 [at] llnl.gov.

January 13, 2010

• Obtain source code at https://wci.llnl.gov/sites/wci/files/2021-01/silo-4.7.2.tgz
• Obtain documentation at https://wci.llnl.gov/content/assets/docs/simulation/computer-codes/silo/LLNL_SM_421083.pdf

This release is to correct a couple of critical problems with the 4.7.1 release. The 4.7.1 release has been removed from the web site and uninstalled from LLNL systems. Again, apologies for the problems in the 4.7.1 release. This release has been tested and valgrind'd against its own test suite as well as with VisIt 1.12 and VisIt 2.0.

Silo 4.7.2 is link-time compatible with 4.7.1 except for codes using DB_LONG_LONG. This is because the way DB_LONG_LONG was being handled in the Silo header file had to change to correct a problem with it.

Critical bugs fixed in this release:

• DB_LONG_LONG is no longer mapped to DB_LONG on platforms where sizeof(long)==sizeof(long long).
• The problem where centering for Quadvars was always being treated as node centered was corrected.
• A UMR in handling of arrays of strings in multi-block objects was corrected.

Other bugs fixed in this release:

• Diffing data of different type in browser is now commutative.
• Default formats for printing long and long long data in base 8/16 were corrected.
• Missing diffing parameters for long long type were added.
• Some leaks and a memory error were corrected in a couple of tests.
• Overlap of src/dst in a strcpy in PDB was fixed.

January 6, 2010

Silo 4.7.1 was found to have a couple of serious bugs and has been removed from the site. A 4.7.2 release will be made by 1/12/10.

December 10, 2009

• Obtain source code at https://wci.llnl.gov/content/assets/docs/simulation/computer-codes/silo… ***Removed 1/6/10
• Obtain documentation at https://wci.llnl.gov/content/assets/docs/simulation/computer-codes/silo/LLNL_SM_421083.pdf
• HPC Podcast at www.rce-cast.com on the topic of Silo

Organization of information in this release announcement

1. New features added in this release
2. Bugs fixed in this release
3. Details regarding long long support
4. Details on new, alternative relative differencing scheme in browser'

1. New features added in this release

• Support for DB_LONG_LONG datatype was added.
• A new relative differencing option was added to silodiff/browser.
• Species now support names and colors just as materials do.
• All Silo structs are now non-anonymous.
• All silo variables now support characterization as conserved quantities.
• All silo variables now support characterization as extensive quantities.
• Browser can now display the contents of the HDF5 driver's (magic) '/.silo' directory.
• Hard links are now supported by the friendly HDF5 names feature.
• FPZIP and HZIP support is now included for Windows platforms.
• silodiff now warns of possible mismatch with browser path.
• silodiff/browser now supports differing values of different primitive type.

2. Bugs fixed in this release

• Problems writing edge- and face-centered quadvars and ucdvars were fixed.
• silock now works for data produced on HDF5 driver.
• silodiff corrrectly sets $lowlevel variable so that HDF5 file differences cannot be missed.
• browser now displays mixvals for ucdvars.
• libsilo.a|so no longer has dependence on libreadline.
• Fortran interface now correctly compiles for Windows.
• A double-free memory error causing silex to crash was fixed.
• A missing pmpio.h include file dependency was added.
• Browser now outputs version of Silo library it is linked with.
• A problem where once compression features were enabled, they could never be disabled was corrected.
• Multi-part, multi-block objects written on Windows but read on Linux and vice-versa now correctly handle the difference in slash character for filesystem pathnames.
• A problem where silo.h and config.h were being included in source distribution tarballs in spite of fact that these files are generated from their .in equivalents was corrected.
• A bug in DBInqFile where it would return true for any PDB file, even a PDB file NOT produced by Silo was corrected. It now returns true ONLY if it finds tell-tale signs there are Silo objects in the file and false otherwise.
• DBCLOSE Fortran wrapper was corrected to set closed dbid to -1.
• Silo's magic file was corrected so that '!' character in PDB moniker is not inadvertently treated as the not-equal operator.

3. Details regarding long long support

• Silo supports a new DBdatatype enum, DB_LONG_LONG.
• Both the PDB and HDF5 drivers were enhanced to support long long datatype. To the extent possible, the lite version of PDB built-in to Silo was modified to match how PDB proper handles long long support. pdbview should display Silo's long long data correctly. However, be aware that fundamental aspects of the lite version of PDB built-in to Silo were changed to support long long.
• On systems where sizeof(long long)==sizeof(long), the DB_LONG_LONG enum resolves to DB_LONG.
• Data written on a system where sizeof(long long)>sizeof(long) and later read on a system where sizeof(long long)==sizeof(long) will work on the HDF5 driver if the values in the long long input can be cast to long without loss of precision. Behavior on the PDB driver in this case is yet to be determined.
• When diffing both the left and right operands of long long type, browser uses the same difference algorithm it has always used but adjusted to support the full integral precision of long long data. In all other cases (and has always been the case), browser casts values to double and computes their differences as double values.

4. Details on new, alternative relative differencing scheme in browser'

• A new relative diffrencing option was added to browser. Thanks to John Wholbier for specifying the algorithm. It is controlled by the -x EPS, --epsilon=EPS command line arguments and $diff_*_eps variables.
• Two numbers, A and B, are different if |A-B|/(|A|+|B|+EPS)>RTOL where EPS is the epsilon parameter and RTOL is the relative difference threshold.
• For EPS=0, the algorithm is similar (but not identical) to normal relative differencing. But for EPS=1, it behaves in such a way as to shift between this alternate relative differencing for large numbers and absolute differencing for numbers near zero. Default is -1 (e.g., turned off).
• By default, EPS is set to -1 so it is disabled. However, setting it to any positive value enables this alterantive differencing algorithm.

February 20, 2009

Silo has a smaller tarball for 4.7 now available for your use. Here is a smaller silo-4.7-smalltest.tgz tarball to use.

January 15, 2009

Obtain source code here: https://wci.llnl.gov/sites/wci/files/2021-01/silo-4.7.tgz

Obtain documentation here: https://wci.llnl.gov/content/assets/docs/simulation/computer-codes/silo/LLNL-SM-410226.pdf

Organization of information in this announcement.

1. New features in this release
2. Bugs fixed in this release
3. Details regarding API change for datatype'd pointers
4. Other general information

A. New features in this release

• The API was modernized in places where the datatype of an array is specified using 'int datatype' to use 'void*' in place of 'float*'.
• This API change may be 'turned off' at configure time and will otherwise lead to compiler errors requiring the client to correctly cast a pointer. Read item C, below for more details.
• A majority of the test suite now compiles and runs on Windows. Thanks to Kathleen Bonnell for doing this.
• Namescheme functions were added to support on-demand generation of names of arrays of sets in mrgtrees.
• Version info functions were added to obtain and/or compare versions of the library the executable is linked with as well as versions of the library a given file was created with.
• Example files and source code for using Silo for block-structured AMR meshes in 2D and 3D have been developed.
• silodiff now supports diff'ing directories as well as making it easy and less error prone to diff a file with another by the same name in a different directory.
• silex was was ported to (and now requires) Qt-4. Thanks to Jeremy Meredith for doing this.
• User's manual was updated to match the current, 4.7 API.

B. Bugs fixed in this release.

• The DBOPT_TOPO_DIM option for ucd meshes was being handled incorrectly defaulting always to set topological dimension to be equal to spatial dimension.
• A user reported 'db_close' public symbol name collides with BRL/CAD making it impossible to link the two in the same application. That symbol name was changed.
• Numerous tests failed to compile on windows due to large allocations of local variables. Those have been fixed.
• Mrgtrees were not handling sets defined with nameschemes correctly. Attempting to write an Mrgtree using nameschemes was causing a segv. That has been fixed.
• Mrgtrees with nameschemes could not be read correctly. That is fixed.
• Groupel maps with implicit segment ids were causing segvs. Those have been fixed.
• Attempting to write Mrgtree variables (mrgvars) with regions involving nameschemes would segv. Those have been fixed.
• PutMultimeshadj was writing nodelist information to its zonelist member. Thanks to Sean Ahern for fixing this.
• Improved output from browser for mrgtrees.
• Fixed bug in browser output causing extraneous extra blank lines to be output as well as line splitting of arrays of strings that would otherwise fit on a single line.
• Fixed bug in silex preventing it from displaying certain components of objects and spewing error messages to stderr about it.
• A number of leaks in the PDB driver were fixed and more extensive leak testing was performed.

C. Details of API changes from 'float *' to 'void *'

First, have no fear, if you REALLY DON'T WANT TO DEAL WITH THIS, then just make sure you configure Silo with --enable-legacy-datatyped-pointers.

It has been a source of confusion to many users of Silo for many years that functions like DBPutUcdmesh accept an 'int datatype' argument to specify the datatype of the coordinate arrays but than the coordinate arrays themselves are specified as 'float *coords[]' in the API. Users either discovered on their own or asked Silo developers if it is OK to pass pointers to arrays other than float. Indeed, it has been. The same has also been true of the struct objects that Silo returns in its read interface. For example a DBucdmesh object contains an 'int datatype' member to indicate the datatype of the 'coords' member. However, the coords member type was still float* leading to still other confusion when trying to read Silo data. This problem has existed throughout most of the older functions and objects of Silo's API but has been avoided with newer functions and objects by appropriate use of 'void*'. Nonetheless, this behavior for older functions and objects has often led to questions that would have to be fielded by Silo developers.

Starting in version 4.7, all these cases have been 'modernized' to use 'void *'. But, this can be controlled during configuration of the Silo library using the --enable-legacy-datatyped-pointers. This way, if you are comfortable using the old (legacy) conventions, you are free to continue doing so. Support for this will never be removed from the Silo library. However, new users of Silo will most likely compile Silo without this option and experience a less confusing interface. Regardless, the Silo User's manual documents only the modernized interface.

Note that the effect of these API changes are confined primarily to the read half of Silo's interface. So, if you are simply writing Silo files but not reading them, you should experience no errors. Furthermore, on the read half of the interface, the issues you will encounter are compiler errors when you access a void* data member of a Silo object without properly casting it to the desired type, which is specified by an associated 'int datatype' member of the object. Also, since any application reading double precision data would have had to deal with this already to avoid compiler warnings in assigning float* to double*, and since those warnings will become outright errors, the cases that need any attention to be made compliant with the modernized API should be obvious and easily corrected.

D. Other general information

Silo 4.7 is NOT link-time compatible with any previous versions. To use it, you will need to re-compile any Silo client code.

August 25, 2008

Obtain source code here: https://wci.llnl.gov/sites/wci/files/2021-01/silo-4.6.2.tgz

Obtain documentation here: https://wci.llnl.gov/content/assets/docs/simulation/computer-codes/silo/LLNL-SM-406516.pdf

Organization of information in this announcement.

1. New features in this release
2. Other general information
3. Bugs fixed in this release
4. Items specific to LLNL installations

A. New features in this release

• A Silo website has been created (Thanks to Chelle Clements): silo.llnl.gov
• 'silofile' magic file format identification tool. This is just like 'file' command on most Linux systems. In fact, it piggy-backs off of the 'file' command to do its job. It will report whether a given silo file is an HDF5 format or PDB format file. It does so using magic numbers. In fact, 'silofile' can be applied to any file and if the file is not determined to be a Silo file, it will return the same result as the 'file' command.
• New DBCpDir() method to copy whole directory hierarchies from one Silo file to another -- works only on HDF5 driver.
• Integration of Peter Lindtrom's quad/hex mesh compression, hzip, and [1,2,3]D array compression, fpzip. There are several caveats, so be sure to read documentation on DBSetCompression in Silo user's manual. The compression is integrated using HDF5 filters. We have seen lossLESS compression ratios with hzip as high as 20x for Silo test data!. Your mileage may vary.
• DBSetCompression now also provides controls to specify compression error behavior (fallback to uncompressed or fail completely) as well as a minimum target compression ratio.
• Silo has a basic python module, Silo.so. This was actually written by Jeremy Meredith in 2005 but was not generally advertised or compiled by default and until recently was a bit buggy. To use it in your own installation, you need to use both the --enable-pythonmodule and --enable-shared configuration options to Silo. Also, be aware that if you generate data using the HDF5 driver, none of your data will be visible to the python module if you do not also call DBSetFriendlyHDF5Names(1) when creating your file(s).
• Note that there is also another python interface for silo called Pylo developed by Andreas Kl�cknr and you can find it from the Silo we pages.
• Improved testing including inter-format testing, valgrind testing, testing of the pythonmodule, and of browser.
• Improved error messages passed to user-defined error handlers.

B. Other general information

• Silo 4.6.2 is link-time compatible with 4.6.1 and 4.6
• To use HDF5-1.8.1 with Silo 4.6.2 (or any previous version of Silo for that matter), DO NOT configure HDF5 with the --disable-deprecated-symbols configuration flag or the --with-default-api-version=v16 flag.
• To the best of our ability, we have confirmed that the files produced by Silo operating on HDF5-1.6.6 and HDF5-1.8.1 are both backwards and forwards format compatible. That is, applications linked with HDF5-1.6.6 can read files generated by Silo on HDF-1.8.1 and applications linked with HDF5-1.8.1 can read files generated by Silo on HDF5-1.6.6.
• The silo library is now a combination of C and C++ code. The C++ code was added to support new compression features. Rest assured, they can be disabled and the result is the C-only library as you have always known it. However, with these new compression features turned on, be aware that you may need to use a C++ linker to link your application with the Silo library. Alternatively, you may enable shared library builds and avoid having to change your linker.
• You may obtain source code here: https://wci.llnl.gov/sites/wci/files/2021-01/silo-4.6.2.tgz
• You may obtain user's manual here: https://wci.llnl.gov/codes/silo/media/pdf/LLNL-SM-406516.pdf

C. Bugs fixed in this release.

• Ensure browser's output on stdout is line buffered and not unbuffered. When using browser to detect differences in files as in regression testing for example, in unbuffered mode with I/O re-direction of the form 1>foo 2>&1, browser's output was potentially un-predictable. Setting stdout to line buffering should fix this.
• Use browser's $truncate variable to control truncation of string-valued variables simple variables. Previously, browser would summarily truncate any string valued variable to 256 characters.
• Prevent DBClose from throwing a SEGV when closing a file to which a single ucdvar has been written.
• Detect and warn of an attempt to operate on a Silo file that has either been closed or was never opened. Previously, behavior was undetermined and most likely catastrophic with no indication as to cause.
• Detect and warn of attempt to open same file multiple times and not all being read-only mode. Previously, behavior was undetermined and most likely catastrophic with no indication as to cause.
• Python module would core with invalid mem writes after obtaining/printing table of contents (GetToc) or closing a file.
• Diffing of DBcurve objects could result in extraneous output due to opaque driver-specific data being exposed in the object.
• Various memory errors detected by valgrind in PDB-lite.
• Fixed problems compiling python module on several platforms.
• Fixed problems where python module would return strings with embedded trailing null character.
• Test suite now compiles and runs on windows (Thanks to Kathleen Bonnell).
• Various typos and missing information from documentation.
• Fortran compilation on Mac.

D. Items specific to LLNL installations

• Silo 4.6.2 is installed on OCF/SCF at /usr/gapps/silo/4.6.2/$SYS_TYPE
• $SYS_TYPE used to identify installations with caveat that where nondefault compilers have been used, $SYS_TYPE has appended to it the C compiler's name. For example, on yana, chaos_4_x86_64_icc. gcc is the default compiler on all systems except on Aix where it is xlc and BG/L where it is blrts_xlc. Remember, if you need more details regarding a particular installation of silo, read the libsilo.settings file 'next to' libsilo.a or libsilo.so.
• Note that a new 'libso' directory exists for keeping libsiloh5.so and Silo.so separate from libsiloh5.a. So, to get to the pythonmodule shared libs, you will need the 'libso' directory instead of 'lib'
• In addition to the libsilo.a and libsiloh5.a that are typically installed, there is a new libsiloxx.a. Note that libsilo.a and libsiloh5.a DO NOT contain any of the new compression features. Only libsiloxx.a does. libsiloxx.a is a 'maximally feature rich' installation which in addition to new compression features, also includes HDF5 driver. To link with libsiloxx.a, you will most likely need to use a C++ linker.
• All LLNL installations have been made with HDF-1.8.1
• Silo's simple python module, Silo, is now installed on most systems.
• Source tarball is available on LC platforms at: /usr/gapps/silo/4.6.2/silo-4.6.2.tar.gz or https://wci.llnl.gov/sites/wci/files/2021-01/silo-4.6.2.tgz
• User's manual is available on LC platforms at: /usr/gapps/silo/4.6.2/Silo.book.pdf or https://wci.llnl.gov/codes/silo/media/doc/LLNL-SM-406516.pdf

June 1, 2008

Silo announces the deployment of its new website.

January 15, 2008

Current production version: Silo-4.6.1