Build a reproducible scientific workflow

What you can do

Workflow Automation combines scientific data management, numerical processing, optimization and plotting. Define the flow once; SIGCLEAR schedules dependent work and rebuilds affected outputs when inputs or definitions change.

Before starting, install and activate SIGCLEAR and check sglicense status.

Quick start

Start a simple experiment in SIGCLEAR

Create an empty project directory and add a file named SConstruct:

from experiment import *

Process('raw', None,
    '''
    sgcreate nx=10 data.ns=500
    | sgfieldmath offset:f="50*ix"
      data="sinc(30*(data.time-(600^2+offset^2)^0.5/1500))"
    ''')

Process('processed', 'raw',
    'sgfieldmath data="2*data"')

Figure('./processed.png', 'processed',
    'sgplotps')

Run the experiment from that directory:
```
scons
```

The workflow creates the dataset data/raw.sg, transforms it into data/processed.sg, and generates processed.png as a quality-control figure.

Essential concepts

Process: connects source files, a Program flow and output targets.
Figure: a Process that produces a dependency-aware QC figure.
Program: an executable performing one processing operation.
Package: related Programs, libraries or APIs installed together.

Datasets normally use data/ and the .sg suffix. Choose an input schema, units, grouping and acceptance checks before adapting a workflow. Validate a small known case before scaling up.

Continue your experiment

ExperimentProcesses, Figures and local/HPC execution SOIGCreate, inspect and organize scientific datasets I/OExchange table, binary and MATLAB data DSPTransform, filter and interpolate fields OptimizationFit models and solve inverse problems PlotGenerate scientific figures

Browse the worked examples for complete import, processing and quality-control workflows. Consult common Program options for shared controls such as --help and --nthread.