Examples¶

This example provides usage examples of SWPy's data collection module.

SWPy SWDS Request¶

swpy.swds.request Module¶

Request module provides functionality to search and download SWPy data.

Main Class & Functions¶

`data_info`( )	Function of sequentially searching by selecting list.
`search`(table_name, data_id, start_time, end_time=None)	Function of searching by specifying the desired data and period.
`Downloader`	Class for downloading desired data.

Examples of Request Module¶

Here are some examples of using class and functions to search and save data.

Import the data collection module into Python.

from swpy.swds.request import data_info, search, Downloader

data_info()¶

Call the data_info function.

>>> data_info()

Invoking the function will display the following table list and input field.

  -----Table List-----
ghn
kasi_boh_asc
kasi_dst_prediction
kasi_gye_scint
kasi_gye_vhf
kasi_hq_escallisto
kasi_jbs_asc
kasi_jbs_scint
kasi_ksrbl
kasi_magnetometer
kopri_jbs_asc
kopri_jbs_scint
nasa_sdo
nasa_soho
noaa_ace
noaa_goes
noaa_swpc
wdc_dst
Enter the table number / exit:        # Input field

a. Enter the desired 'table number' in the input field.

b. Enter 'exit' if you want to terminate the data search.

Entering the table number will display a list of data ID

# Example of selecting the 'ghn'
 -----Data ID List-----
1: ghn_bbso
2: ghn_gong
3: ghn_kanz
4: ghn_oact
Enter the data id number / exit / undo:        # Input field

a. Enter the desired 'data number' in the input field.

b. Enter 'exit' if you want to terminate the data search.

c. Enter 'undo' if you want to return to the previous page.

Entering the data number will display the information for the data

# Example of selecting the 'ghn_bbso'
 -----Data Info-----
table: ghn
data_id: ghn_bbso
start_time: 2010-01-05
end_time: 2024-07-15
searchable_timeframe: 7 days
institute: None
observatory: None
satellite: None
model: None
telescope: None
wavelength: None
channel: None
instrument: None
Enter the start time (YYYYMMDD) / exit / undo:        # Input field

a. Enter the desired 'start time' in the input field.

* If you do not follow the YYYY-MM-DD format, you will not obtain the desired data.

* Additionally, the searchable_timeframe is fixed at 7 days for all data, so you can only fetch up

to a maximum of 7 days of data.

Entering the start time will display the information for the data list

# Example of entering the '2020-01-01'
 -----Data List-----
data_start_time: 2020-01-01
data_end_time: 2020-01-08      # Automatically fetch up to 7 days of data
data_list:
time: 2020-01-02 03:47:42 / name: bbso_halph_fl_20200101_184742.fts
time: 2020-01-02 03:47:42 / name: bbso_halph_fl_20200101_184742.jpg
                            ...
time: 2020-01-08 05:39:33 / name: bbso_halph_fr_20200107_203933.jpg
time: 2020-01-08 05:39:33 / name: bbso_halph_fr_20200107_203933.fts
Enter the end time (YYYYMMDD) / exit / undo:          # Input field

a. Enter 'end time' to specify the desired period.

b. Enter 'exit' if you want to terminate the data search.

c. Enter 'undo' if you want to return to the previous page.

d. Enter 'skip' in the input field the end time will automatically be set to 7 days later, and then

the data list will be displayed.

e. Enter 'download' it will move to the download page.

Entering the end time or skip will display the information for the desired period data

# Example of selecting the '2020-01-03'
 -----Data List-----
data_start_time: 2020-01-01
data_end_time: 2020-01-03
data_list:
time: 2020-01-02 03:47:42 / name: bbso_halph_fl_20200101_184742.jpg
time: 2020-01-02 03:47:42 / name: bbso_halph_fl_20200101_184742.fts
time: 2020-01-02 05:55:24 / name: bbso_halph_fl_20200101_205524.fts
                        ...
time: 2020-01-03 04:59:55 / name: bbso_halph_fr_20200102_195955.fts
time: 2020-01-03 04:59:55 / name: bbso_halph_fr_20200102_195955.jpg
Enter the download / exit / undo:                     # Input field

a. Entering 'exit' or 'undo' serves the same purpose as in the previous steps.

search(table_name, data_id, start_time, end_time=None)¶

Assign search to the variable as follows.

>>> d = search(table_name = 'ghn', data_id = 'ghn_bbso', start_time = '2020-01-01', end_time = '2020-01-03')

The green text is an example of input values. Enter the desired data values in the corresponding location.

By assigning search to the variable, it will display a list of data within the specified period.

data_start_time: 2020-01-01
data_end_time: 2020-01-03
data_list:
time: 2020-01-02 03:47:42 / name: bbso_halph_fl_20200101_184742.fts
time: 2020-01-02 03:47:42 / name: bbso_halph_fl_20200101_184742.jpg
                        ...
time: 2020-01-03 04:59:55 / name: bbso_halph_fr_20200102_195955.jpg
time: 2020-01-03 04:59:55 / name: bbso_halph_fr_20200102_195955.fts

The list of data within the specified period will be assigned to the Downloader.

Downloader¶

Enter the local path to download the data list assigned from search.

>>> d.download(r'file/path/data.zip')

Specify the folder name with a .zip extension at the end of the path to save the file.

* Only .zip format, is supported.

* If a .zip folder is not specified in the path, the download will not occur.

SWPy SWDS Dataset¶

swpy.swds.dataset Module¶

Dataset module provides functionality to to check and preprocess SWPy data.

Main Class & Functions¶

`data_local`(filepath, keyword)	Function of reading data from the local drive.
`Processor`	Class that reads data from the database in conjunction with `swds.request.request`.
`db_search`( )	Function to load a file from the database and store it in a variable.

Examples of Dataset Module¶

Here are some examples of using class and functions to show SWPy data information and plot graphs.

Import the data collection module into Python.

from swpy.swds.request import data_info, search, Downloader
from swpy.swds.dataset.utilities import Processor, data_local
from swpy.swds.dataset.utilities import BasePlot
from swpy.swds.dataset.goes import goes

'ace' or 'discovr' can be input istead of 'goes'

Processor¶

Use the search function from the swpy.swds.request module.

# Example of goes dataset
>>> search = search(table_name = 'noaa_goes', data_id = 'noaa_swpc_gp_geomag_1m', start_time = '2024-01-01')

table_name and table_id can be found in data_info.

To read data from the Database and assign it to the db_search function.

>>> data = Processor(search)

db_search(self)¶

Using the dataset information from the Processor , find the file in the Database, read it, and store it in the data_all variable.

>>> data_all = data.db_search()

If you print data_all , you can check the data stored as a numpy array.

>>> print(data_all)

data_start_time: 2024-01-01
data_end_time: 2024-01-08
data_list:
time: 2024-01-01 09:00:00 / name: 20240101_Gp_mag_1m.txt
time: 2024-01-02 09:00:00 / name: 20240102_Gp_mag_1m.txt
                        ...
time: 2024-01-07 09:00:00 / name: 20240107_Gp_mag_1m.txt
time: 2024-01-08 09:00:00 / name: 20240108_Gp_mag_1m.txt
[[':Data_list: 20240101_Gp_mag_1m.txt']
 [':Created: 2024-01-01 23:59 UTC']
 ['# Prepared by Korea Astronomy and Space Science Institute (KASI), Space Weather Research Center']
 ...
 ['2024 01 08  2359\t16\t\t78.74858856201172        \t18.5080509185791         \t0.8024502992630005       \t80.8987045288086         \t0']]

If the data period is 7 days or less, the unfilled arrays are filled with NaN values.

Check the dataset information stored in data_all and plot it.

>>> dataset = goes.goes_open(data_all)
>>> dataset.info()
>>> dataset.plot()

Example of dataset.info

Institute:       NOAA
Observatory:     GOES
Satellite:       GOES-16
Instrument:      MAG
Telescope:       None
Detector:        MAG
Model:           None
Object:          Magnetic Field
Date:            Start date: 2024-01-01, End date: 2024-01-08
Dimension:       (11520,)
Data num:        11520

The plot function is explained in detail below.

data_local(filepath, keyword)¶

Enter the filepath and the keyword for the observation satellite recorded in the file name.

>>> data_all = data_local(r'file/path/ ', 'goes')

When the list of data matching the keyword is loaded from the filepath, the start day input field and the data period input field appear in sequence.

Input start day (YYYYmmdd): 
Setting for the total plot day (1-7): 

Checking the dataset information and plotting it is the same as for db_search.

>>> dataset = goes.goes_open(data_all)
>>> dataset.info()
>>> dataset.plot()

plot(self, start_time=None, end_time=None, target=None, sep_plots=True, figsize=None, log=True)¶

The plot function embedded in swpy.swds.utilities.BasePlot , responsible for the dataset's plotting functionality.

Six basic features are defined within the function.

a. If you want to plot a specific time period, enter the start_time and end_time.

b. If you want to plot only the desired observation targets, enter the desired targets as a list in target.

c. If you want to combine the observation targets into one main plot, set sep_plots = False.

d. If you want to adjust the plot size, enter the desired dimensions in figsize . The default size is (12, 6).

e. If you want to plot the data on a linear scale, set log = False.

>>> dataset.plot(start_time = '2024-01-02 00:00:00', end_time = '2024-01-06 00:00:00', target = ['Hp','He'] sep_plots = False figsize = (20, 9) log = False)