Write, View, and Load Data with a JSON
PowerSystems.jl
provides functionality to serialize an entire System
to a JSON file and then deserialize it back to a System
. The main benefit is that deserializing is significantly faster than reconstructing the System
from raw data files.
The sections below show how to write data to a JSON, explore the data while it is in JSON format, and load Data saved in a JSON back into PowerSystems.jl
.
Write data to a JSON
You can do this to save your own custom System
, but we'll use an existing dataset from PowerSystemCaseBuilder.jl
, simply to illustrate the process.
First, load the dependencies and a System
from PowerSystemCaseBuilder
:
julia> using PowerSystems
julia> using PowerSystemCaseBuilder
julia> sys = build_system(PSISystems, "c_sys5_pjm")
┌ Info: Building new system c_sys5_pjm from raw data └ sys_descriptor.raw_data = "/home/runner/.julia/artifacts/afb608473cf4d5eb22147856de1a1a651f36d40b/PowerSystemsTestData-3.1/psy_data/data_5bus_pu.jl" [ Info: Serialized time series data to /home/runner/.julia/packages/PowerSystemCaseBuilder/uZO8H/data/serialized_system/614e094ea985a55912fc1321256a49b755f9fc451c0f264f24d6d04af20e84d7/c_sys5_pjm_time_series_storage.h5. [ Info: Serialized System to /home/runner/.julia/packages/PowerSystemCaseBuilder/uZO8H/data/serialized_system/614e094ea985a55912fc1321256a49b755f9fc451c0f264f24d6d04af20e84d7/c_sys5_pjm.json [ Info: Serialized System metadata to /home/runner/.julia/packages/PowerSystemCaseBuilder/uZO8H/data/serialized_system/614e094ea985a55912fc1321256a49b755f9fc451c0f264f24d6d04af20e84d7/c_sys5_pjm_metadata.json System ┌───────────────────┬─────────────┐ │ Property │ Value │ ├───────────────────┼─────────────┤ │ Name │ │ │ Description │ │ │ System Units Base │ SYSTEM_BASE │ │ Base Power │ 100.0 │ │ Base Frequency │ 60.0 │ │ Num Components │ 27 │ └───────────────────┴─────────────┘ Static Components ┌───────────────────┬───────┐ │ Type │ Count │ ├───────────────────┼───────┤ │ ACBus │ 5 │ │ Arc │ 6 │ │ Line │ 6 │ │ PowerLoad │ 3 │ │ RenewableDispatch │ 2 │ │ ThermalStandard │ 5 │ └───────────────────┴───────┘ Time Series Summary ┌───────────────────┬────────────────┬──────────────────┬─────────────────────── │ owner_type │ owner_category │ time_series_type │ time_series_category ⋯ │ String │ String │ String │ String ⋯ ├───────────────────┼────────────────┼──────────────────┼─────────────────────── │ PowerLoad │ Component │ SingleTimeSeries │ StaticTimeSeries ⋯ │ RenewableDispatch │ Component │ SingleTimeSeries │ StaticTimeSeries ⋯ └───────────────────┴────────────────┴──────────────────┴─────────────────────── 3 columns omitted
Set up your target path, for example in a "mysystems" subfolder:
julia> folder = mkdir("mysystems");
julia> path = joinpath(folder, "system.json")
"mysystems/system.json"
Now write the system to JSON:
julia> to_json(sys, path)
[ Info: Serialized time series data to mysystems/system_time_series_storage.h5. [ Info: Serialized System to mysystems/system.json [ Info: Serialized System metadata to mysystems/system_metadata.json
Notice in the Info
statements that the serialization process stores 3 files:
- System data file (
*.json
file) - Validation data file (
*.json
file) - Time Series data file (
*.h5
file)
Viewing PowerSystems
Data in JSON Format
Some users prefer to view and filter the PowerSystems.jl
data while it is in JSON format. There are many tools available to browse JSON data.
Here is an example GUI tool that is available online in a browser.
The command line utility jq offers even more features. Below are some example commands, called from the command line within the "mysystems" subfolder:
View the entire file pretty-printed:
jq . system.json
View the PowerSystems
component types:
jq '.data.components | .[] | .__metadata__ | .type' system.json | sort | uniq
View specific components:
jq '.data.components | .[] | select(.__metadata__.type == "ThermalStandard")' system.json
Get the count of a component type:
# There is almost certainly a better way.
jq '.data.components | .[] | select(.__metadata__.type == "ThermalStandard")' system.json | grep -c ThermalStandard
View specific component by name:
jq '.data.components | .[] | select(.__metadata__.type == "ThermalStandard" and .name == "107_CC_1")' system.json
Filter on a field value:
jq '.data.components | .[] | select(.__metadata__.type == "ThermalStandard" and .active_power > 2.3)' system.json
Read the JSON file and create a new System
Finally, you can read the file back in, and verify the new system has the same data as above:
julia> sys2 = System(path)
[ Info: Loaded time series from storage file existing=mysystems/system_time_series_storage.h5 new=/tmp/jl_Xi2rnv compression=CompressionSettings(false, CompressionTypes.DEFLATE = 1, 3, true) System ┌───────────────────┬─────────────┐ │ Property │ Value │ ├───────────────────┼─────────────┤ │ Name │ │ │ Description │ │ │ System Units Base │ SYSTEM_BASE │ │ Base Power │ 100.0 │ │ Base Frequency │ 60.0 │ │ Num Components │ 27 │ └───────────────────┴─────────────┘ Static Components ┌───────────────────┬───────┐ │ Type │ Count │ ├───────────────────┼───────┤ │ ACBus │ 5 │ │ Arc │ 6 │ │ Line │ 6 │ │ PowerLoad │ 3 │ │ RenewableDispatch │ 2 │ │ ThermalStandard │ 5 │ └───────────────────┴───────┘ Time Series Summary ┌───────────────────┬────────────────┬──────────────────┬─────────────────────── │ owner_type │ owner_category │ time_series_type │ time_series_category ⋯ │ String │ String │ String │ String ⋯ ├───────────────────┼────────────────┼──────────────────┼─────────────────────── │ PowerLoad │ Component │ SingleTimeSeries │ StaticTimeSeries ⋯ │ RenewableDispatch │ Component │ SingleTimeSeries │ StaticTimeSeries ⋯ └───────────────────┴────────────────┴──────────────────┴─────────────────────── 3 columns omitted
PowerSystems generates UUIDs for the System
and all components in order to have a way to uniquely identify objects. During deserialization it restores the same UUIDs. If you will modify the System
or components after deserialization then it is recommended that you set this flag to generate new UUIDs.
system2 = System(path; assign_new_uuids = true)