Date-Time Cycling§

In the last section we looked at writing an integer cycling workflow, one where the cycle points are numbered.

Reminder

Cycle points are labels. Cylc runs tasks as soon as their dependencies are met so cycles do not necessarily run in order.

ISO8601§

In Cylc, dates, times and durations are written using the ISO8601 format - an international standard for representing dates and times.

Date-Times§

Date-Times§

Warning

The “basic” (purely numeric except for T) and “extended” (written with hyphens and colons) formats cannot be mixed. For example the following date-times are invalid:

2000-01-01T0000
20000101T00:00

Durations§

Y for year.
M for month.
D for day.
W for week.
H for hour.
M for minute.
S for second.

Durations§

P1D: one day.
PT1H: one hour.
P1DT1H: one day and one hour.
PT1H30M: one and a half hours.
P1Y1M1DT1H1M1S: a year and a month and a day and an hour and a minute and a second.

Date-Time Recurrences§

In integer cycling, suites’ recurrences are written P1, P2, etc.

In date-time cycling there are two ways to write recurrences:

Using ISO8601 durations (e.g. P1D, PT1H).
Using ISO8601 date-times with inferred recurrence.

Inferred Recurrence§

2000-01-01T00   # Datetime - midnight on the 1st of January 2000.

     01-01T00   # Every year on the 1st of January.
        01T00   # Every month on the first of the month.
          T00   # Every day at midnight.
          T-00  # Every hour at zero minutes past (every hour on the hour).

Note

To omit hours from a date time we must place a - after the T character.

Recurrence Formats§

By defining an arbitrary cycle point (datetime/recurrence):
- 2000/P1Y: every year starting with the year 2000.
- 2000-01-01T00/T00: every day at midnight starting on the 1st of January 2000
- 2000-01-01T12/T00: every day at midnight starting on the first midnight after the 1st of January at 12:00 (i.e. 2000-01-02T00).

Recurrence Formats§

By defining an offset from the initial cycle point (offset/recurrence). This offset is an ISO8601 duration preceded by a plus character:
- +P1Y/P1Y: every year starting one year after the initial cycle point.
- +PT1H/T00: every day starting on the first midnight after the point one hour after the initial cycle point.

Durations And The Initial Cycle Point§

When using durations, beware that a change in the initial cycle point might produce different results for the recurrences.

Durations And The Initial Cycle Point§

[scheduling]
    initial cycle point = 2000-01-01T00
    [[dependencies]]
        [[[P1D]]]
            graph = foo[-P1D] => foo

[scheduling]
    initial cycle point = 2000-01-01T12
    [[dependencies]]
        [[[P1D]]]
            graph = foo[-P1D] => foo

Durations And The Initial Cycle Point§

We could write the recurrence “every midnight” independent from the initial cycle point by:

Use an inferred recurrence instead (i.e. T00).
Overriding the recurrence start point (i.e. T00/P1D)
Using the [scheduling]initial cycle point constraints setting to constrain the initial cycle point (e.g. to a particular time of day). See the Cylc User Guide for details.

The Initial & Final Cycle Points§

R1: repeat once at the initial cycle point.
R1/P0Y: repeat once at the final cycle point.

Inter-Cycle Dependencies§

foo[-P1D]: the task foo from the cycle one day before.
bar[-PT1H30M]: the task bar from the cycle 1 hour 30 minutes before.

baz[^]: the task baz from the initial cycle point.

UTC Mode§

[cylc]
    UTC mode = True

Putting It All Together§

We will now develop a simple weather forecasting suite.

1. Gathering Observations§

2. Running computer models to generate forecast data§

3. Processing the data output to produce user-friendly forecasts§

Next Steps

Read through the “Putting It All Together” section.
Complete the practical.

Next section: Further Scheduling

Practical

In this practical we will create a dummy forecasting suite using date-time cycling.

Create A New Suite.

Within your ~/cylc-run directory create a new directory called datetime-cycling and move into it:

mkdir ~/cylc-run/datetime-cycling
cd ~/cylc-run/datetime-cycling

Create a suite.rc file and paste the following code into it:

[cylc]
    UTC mode = True
[scheduling]
    initial cycle point = 20000101T00Z
    [[dependencies]]

Add The Recurrences.

The weather-forecasting suite will require two recurrences. Add sections under the dependencies section for these, based on the information given above.

Hint

See Date-Time Recurrences.
Solution

The two recurrences you need are
- PT3H: repeat every three hours starting from the initial cycle point.
- +PT6H/PT6H: repeat every six hours starting six hours after the initial cycle point.
```
 [cylc]
     UTC mode = True
 [scheduling]
     initial cycle point = 20000101T00Z
     [[dependencies]]
+        [[[PT3H]]]
+        [[[+PT6H/PT6H]]]
```

Write The Graphing.

With the help of the graphs and the information above add dependencies to your suite to implement the weather-forecasting workflow.

You will need to consider the inter-cycle dependencies between tasks.

Use cylc graph to inspect your work.

Hint

The dependencies you will need to formulate are as follows:

The consolidate_observations task is dependent on the get_observations_<site> tasks.
The forecast task is dependent on:
- the get_rainfall task;
- the consolidate_observations tasks from:
  - the same cycle;
  - the cycle 3 hours before (-PT3H);
  - the cycle 6 hours before (-PT6H).
The post_process_exeter task is dependent on the forecast task.

To launch cylc graph run the command:

cylc graph <path/to/suite.rc>

Solution

[cylc]
    UTC mode = True
[scheduling]
    initial cycle point = 20000101T00Z
    [[dependencies]]
        [[[PT3H]]]
            graph = """
                get_observations_aldergrove => consolidate_observations
                get_observations_camborne => consolidate_observations
                get_observations_heathrow => consolidate_observations
                get_observations_shetland => consolidate_observations
            """
        [[[+PT6H/PT6H]]]
            graph = """
                consolidate_observations => forecast
                consolidate_observations[-PT3H] => forecast
                consolidate_observations[-PT6H] => forecast
                get_rainfall => forecast => post_process_exeter
            """

Inter-Cycle Offsets.

To ensure the forecast tasks for different cycles run in order the forecast task will also need to be dependent on the previous run of forecast.

We can express this dependency as forecast[-PT6H] => forecast.

Try adding this line to your suite then visualising it with cylc graph.

You will notice that there is a dependency which looks like this:

Note in particular that the forecast task in the 00:00 cycle is grey. The reason for this is that this task does not exist. Remember the forecast task runs every six hours starting 6 hours after the initial cycle point, so the dependency is only valid from 12:00 onwards. To fix the problem we must add a new dependency section which repeats every six hours starting 12 hours after the initial cycle point.

Make the following changes to your suite and the grey task should disappear:
```
           [[[+PT6H/PT6H]]]
               graph = """
                   ...
-                  forecast[-PT6H] => forecast
               """
+          [[[+PT12H/PT6H]]]
+              graph = """
+                  forecast[-PT6H] => forecast
+              """
```