`rose_prune`

A framework for housekeeping a cycling suite.

It prunes files and directories generated by suite tasks. It is designed to work under rose task-run on the host that runs the suite daemon.

The application is normally configured in the rose_prune[prune] section in the rose-app.conf file.

All settings are expressed as a space delimited list of cycles, normally as cycle points or offsets relative to the current cycle. For datetime cycling, the format of a cycle point should be an ISO8601 datetime, and an offset should be an ISO8601 duration. E.g. -P1DT6H is 1 day and 6 hours before the current cycle point.

The cycles of some settings also accept an optional argument followed by a colon. In these, the argument should be globs for matching items in the directory. If two or more globs are required, they should be separated by a space. In which case, either the argument should be quoted or the space should be escaped by a backslash.

Note

rose_prune uses Bash extglob pattern matching which supports simple (e.g. *) and extended (e.g. !(foo)) pattern matching.

For more information see the shopt documentation for the version of bash you have installed ($ man shopt).

Invocation

In automatic selection mode, this built-in application will be invoked automatically if a task has a name that starts with rose_prune*.

Example

meta=rose_prune
mode=rose_prune

[prune]
cycle-format{cycle_year_month}=CCYYMM
prune-remote-logs-at=-PT6H
archive-logs-at=-P1D
prune-server-logs-at=-P7D
prune{work}=-PT6H:task_x* -PT12H:*/other*.dat -PT18H:task_y* -PT24H
prune{share}=-P1D:hello-*-at-%(cycle)s.txt -P3M:monthly/%(cycle_year_month)s/
prune{share/cycle}=-PT6H:foo* -PT12H:'bar* *.baz*' -P1D

Configuration

Rose App rose_prune

Config [prune]

Config cycle-format{key}= format

Specify a key to a format string for use in conjunction with a prune{item-root}=cycle:globs setting. For example, we may have something like cycle-format{cycle_year}=CCYY and prune{share}=-P1Y:xmas-present-%(cycle_year)s/. In Cylc, if the current cycle point is 20151201T0000Z, it will clear out the directory share/xmas-present-2014/.

The key can be any string that can be used in a %(key)s substitution, and format should be a a valid rose date print format.

Config prune-remote-logs-at= cycle ...: Remove remote job logs at these cycles.

Config prune-server-logs-at= cycle ...: Remove logs on the suite server. Removes both log directories and archived logs.

Config archive-logs-at= cycle ...: Archive all job logs at these cycles. Remove remote job logs on success.

Config prune{item-root}= cycle[:globs] ...

Remove the sub-directories under item-root (e.g. work/ of the specified cycles. E.g. In Cylc, if current cycle is 20141225T1200Z, prune{work}=-PT12H will clear out work/20141225T0000Z/.

If globs are specified for a cycle, it will attempt to prune only items matching CYCLE/GLOBS under item-root. E.g. In Cylc, if current cycle is 20141225T1200Z, then prune{share/cycle}=-PT12H:wild* will clear out all items matching share/cycle/20141225T0000Z/wild*.

A glob can also be specified as a formatting string containing a single substitution %(cycle)s. In this mode, the cycle string will not be added as a sub-directory of the item-root. E.g. In Cylc, if current cycle is 20141225T1200Z, then prune{share}=-PT12H:hello-*-at-%(cycle)s.txt will clear out all items matching share/hello-*-at-20141225T0000Z.txt.

A glob can also be specified as a formatting string containing a substitution %(key)s, if a cycle-format{key}=format setting is specified.

Config prune-work-at= cycle[:globs] ...: Deprecated since version foo: Equivalent to prune{work}=cycle[:globs] ....

Config prune-datac-at= cycle[:globs] ...: Deprecated since version foo: Equivalent to prune{share/cycle}=cycle[:globs] ....

foo