Write, View, and Load Data with a JSON

PowerSystems.jl provides functionality to serialize an entire System to a JSON file and then deserialize it back to a System. The main benefit is that deserializing is significantly faster than reconstructing the System from raw data files.

The sections below show how to write data to a JSON, explore the data while it is in JSON format, and load Data saved in a JSON back into PowerSystems.jl.

Write data to a JSON

You can do this to save your own custom System, but we'll use an existing dataset from PowerSystemCaseBuilder.jl, simply to illustrate the process.

First, load the dependencies and a System from PowerSystemCaseBuilder:

julia> using PowerSystems
julia> using PowerSystemCaseBuilder
julia> sys = build_system(PSISystems, "c_sys5_pjm")┌ Info: Building new system c_sys5_pjm from raw data
└   sys_descriptor.raw_data = "/home/runner/.julia/artifacts/afb608473cf4d5eb22147856de1a1a651f36d40b/PowerSystemsTestData-3.1/psy_data/data_5bus_pu.jl"
[ Info: Serialized time series data to /home/runner/.julia/packages/PowerSystemCaseBuilder/uZO8H/data/serialized_system/614e094ea985a55912fc1321256a49b755f9fc451c0f264f24d6d04af20e84d7/c_sys5_pjm_time_series_storage.h5.
[ Info: Serialized System to /home/runner/.julia/packages/PowerSystemCaseBuilder/uZO8H/data/serialized_system/614e094ea985a55912fc1321256a49b755f9fc451c0f264f24d6d04af20e84d7/c_sys5_pjm.json
[ Info: Serialized System metadata to /home/runner/.julia/packages/PowerSystemCaseBuilder/uZO8H/data/serialized_system/614e094ea985a55912fc1321256a49b755f9fc451c0f264f24d6d04af20e84d7/c_sys5_pjm_metadata.json
System
┌───────────────────┬─────────────┐
│ Property          │ Value       │
├───────────────────┼─────────────┤
│ Name              │             │
│ Description       │             │
│ System Units Base │ SYSTEM_BASE │
│ Base Power        │ 100.0       │
│ Base Frequency    │ 60.0        │
│ Num Components    │ 27          │
└───────────────────┴─────────────┘

Static Components
┌───────────────────┬───────┐
│ Type              │ Count │
├───────────────────┼───────┤
│ ACBus             │ 5     │
│ Arc               │ 6     │
│ Line              │ 6     │
│ PowerLoad         │ 3     │
│ RenewableDispatch │ 2     │
│ ThermalStandard   │ 5     │
└───────────────────┴───────┘

Time Series Summary
┌───────────────────┬────────────────┬──────────────────┬───────────────────────
│ owner_type        │ owner_category │ time_series_type │ time_series_category ⋯
│ String            │ String         │ String           │ String               ⋯
├───────────────────┼────────────────┼──────────────────┼───────────────────────
│ PowerLoad         │ Component      │ SingleTimeSeries │ StaticTimeSeries     ⋯
│ RenewableDispatch │ Component      │ SingleTimeSeries │ StaticTimeSeries     ⋯
└───────────────────┴────────────────┴──────────────────┴───────────────────────
                                                               3 columns omitted

Set up your target path, for example in a "mysystems" subfolder:

julia> folder = mkdir("mysystems");
julia> path = joinpath(folder, "system.json")"mysystems/system.json"

Now write the system to JSON:

julia> to_json(sys, path)[ Info: Serialized time series data to mysystems/system_time_series_storage.h5.
[ Info: Serialized System to mysystems/system.json
[ Info: Serialized System metadata to mysystems/system_metadata.json

Notice in the Info statements that the serialization process stores 3 files:

1. System data file (*.json file)
2. Validation data file (*.json file)
3. Time Series data file (*.h5 file)

Viewing PowerSystems Data in JSON Format

Some users prefer to view and filter the PowerSystems.jl data while it is in JSON format. There are many tools available to browse JSON data.

Here is an example GUI tool that is available online in a browser.

The command line utility jq offers even more features. Below are some example commands, called from the command line within the "mysystems" subfolder:

View the entire file pretty-printed:

jq . system.json

View the PowerSystems component types:

jq '.data.components | .[] | .__metadata__ | .type' system.json | sort | uniq

View specific components:

jq '.data.components | .[] | select(.__metadata__.type == "ThermalStandard")' system.json

Get the count of a component type:

# There is almost certainly a better way.
jq '.data.components | .[] | select(.__metadata__.type == "ThermalStandard")' system.json | grep -c ThermalStandard

View specific component by name:

jq '.data.components | .[] | select(.__metadata__.type == "ThermalStandard" and .name == "107_CC_1")' system.json

Filter on a field value:

jq '.data.components | .[] | select(.__metadata__.type == "ThermalStandard" and .active_power > 2.3)' system.json

Read the JSON file and create a new System

Finally, you can read the file back in, and verify the new system has the same data as above:

julia> sys2 = System(path)[ Info: Loaded time series from storage file existing=mysystems/system_time_series_storage.h5 new=/tmp/jl_MRCMK0 compression=CompressionSettings(false, CompressionTypes.DEFLATE = 1, 3, true)
System
┌───────────────────┬─────────────┐
│ Property          │ Value       │
├───────────────────┼─────────────┤
│ Name              │             │
│ Description       │             │
│ System Units Base │ SYSTEM_BASE │
│ Base Power        │ 100.0       │
│ Base Frequency    │ 60.0        │
│ Num Components    │ 27          │
└───────────────────┴─────────────┘

Static Components
┌───────────────────┬───────┐
│ Type              │ Count │
├───────────────────┼───────┤
│ ACBus             │ 5     │
│ Arc               │ 6     │
│ Line              │ 6     │
│ PowerLoad         │ 3     │
│ RenewableDispatch │ 2     │
│ ThermalStandard   │ 5     │
└───────────────────┴───────┘

Time Series Summary
┌───────────────────┬────────────────┬──────────────────┬───────────────────────
│ owner_type        │ owner_category │ time_series_type │ time_series_category ⋯
│ String            │ String         │ String           │ String               ⋯
├───────────────────┼────────────────┼──────────────────┼───────────────────────
│ PowerLoad         │ Component      │ SingleTimeSeries │ StaticTimeSeries     ⋯
│ RenewableDispatch │ Component      │ SingleTimeSeries │ StaticTimeSeries     ⋯
└───────────────────┴────────────────┴──────────────────┴───────────────────────
                                                               3 columns omitted

system2 = System(path; assign_new_uuids = true)