Overview
Scientific and engineering data management
sigclear-soig provides a common dataset structure, library APIs and Programs for creating, inspecting and processing numerical research data.
Quick example
Create and inspect a dataset
sgcreate nx=10 data.ns=500 >example.sg
sgattribute <example.sgThe first command creates 10 columns containing a 500-sample vector. The second displays the resulting dataset structure and attributes.
Self-organized independent grouping
Organize complex measurements into independent groups
A dataset contains independent groups. Each group can contain multiple columns representing sensor or measurement arrays, and every column can hold scalar or vector fields for physical measurements and experimental parameters.
Benefits
Process large datasets consistently
Programs process one independent group at a time, limiting memory requirements for large datasets.
Independent groups can be distributed across computing resources.
Groups, columns and fields provide structure for automatic multipage and multiplot figures.
Custom Programs use a consistent interface across different scientific fields and measurement types.
Included Programs
Create, select and reorganize datasets
| Programs | Purpose |
|---|---|
sgcreate, sgattribute, sgdump | Create datasets and inspect their structure, attributes and values. |
sgwindow, sgsubvol, sgtruncate | Select groups, columns or samples and divide data into processing units. |
sgsort, sgstack, sgpad | Reorder, combine and pad dataset contents. |
sgcat | Concatenate datasets while handling vector sampling and lengths. |
sgfieldin, sgfieldout | Import, select and export fields between processing streams. |
sgfieldtext | Apply supported text and identifier operations to fields. |
Open a Program reference with man PROGRAM, for example man sgcreate.
Libraries and APIs
Access datasets with libsoig
The Package installs libsoig and its public soig.h interface. Custom Programs use the API to read and write groups, columns, fields, attributes and processing history consistently.
Scope
Numerical research data, not business records
SOIG is designed for numerical arrays and measurements produced by sensors, simulations and scientific algorithms. A relational database remains more suitable for customers, orders and other transactional records.