systemslab-server Postprocessing Reference

The server postprocesses completed experiments using the systemslab-dataprocessing package. The default configuration file for postprocessing is found at /usr/share/systemslab-dataprocessing/postprocessing.toml.

The default postprocessing configuration can be overriden either in its entirety or individually by parameter by submitting a custom postprocessing configuration. This can be submitted during experiment creation or from the command line using the systemslab postprocess command. If custom parameters are submitted both during creation and from the CLI command, overrides from the CLI command take precedence.

Options

compression_level

The compression level for the output parquet artifact. If not specified, the default compression level of 8 is used.

tolerance

The maximum skew in system timestamps allowed while aligning entries from different jobs in the same experiment. If not specified, the default tolerance of 200ms is used.

max_row_count

The maximum number of samples in each postprocessed job file. If the actual number of samples is greater than the allowed count, metrics are downsampled. By default, no maximum size is imposed on the artifacts.

groupings

Key-value pairs specifying the columns to use to group data by and the name to be used to provide semantic meaning to the output artifact. An example of a grouping by load, using the ratelimit/current column would look like:

groupings = { load = ["ratelimit/current"] }

service

Configuration for service artifacts, described below.

system

Configuration for service artifacts, described below.

Service and System Metrics

The service and system metrics configurations allow specification of how to find artifacts corresponding to each metric type and how to process them. The parameters are:

drop_histograms

Drop full histograms and only store summary percentiles. Disabled by default.

filter_filename

The filename used to describe the output metric parquet file.

filter_tag

The tag used to describe the output metric parquet file.

pq_conversion

If the file is not a parquet file, use a provided conversion function to convert the data into a parquet. Currently, conversion functions are provided for csv and json files from sysbench.

An example config for systems metrics would look like this

[system]
filter_filename = "metrics.parquet"
filter_tag = "systemslab/metrics"