171 lines
4.2 KiB
Plaintext
171 lines
4.2 KiB
Plaintext
---
|
|
description: Cron Steps are scheduled tasks that run based on cron expressions.
|
|
globs: steps/**/*.step.ts,steps/**/*.step.js,steps/**/*_step.py
|
|
alwaysApply: false
|
|
---
|
|
# Cron Steps Guide
|
|
|
|
Cron Steps enable scheduled task execution using cron expressions.
|
|
|
|
They're typically used for recurring jobs like nightly reports, data synchronization, etc.
|
|
|
|
Cron steps can hold logic, but they do NOT have any retry mechanisms in place, if the logic
|
|
is likely to fail, it's recommended to use CRON Step to emit an event to a topic that will
|
|
ultimately trigger an Event Step that will handle the logic.
|
|
|
|
## Creating Cron Steps
|
|
|
|
Steps need to be created in the `steps` folder, it can be in subfolders.
|
|
|
|
- Steps in TS and JS should end with `.step.ts` and `.step.js` respectively
|
|
- Steps in Python should end with `_step.py`
|
|
|
|
## Definition
|
|
|
|
Defining a CRON Step is done by two elements. Configuration and Handler.
|
|
|
|
### Configuration
|
|
|
|
**TypeScript/JavaScript**: You need to export a config constant via `export const config` that is a `CronConfig` type.
|
|
|
|
**Python**: You need to define a `config` dictionary with the same properties as the TypeScript `CronConfig`.
|
|
|
|
```typescript
|
|
export type Emit = string | {
|
|
/**
|
|
* The topic name to emit to.
|
|
*/
|
|
topic: string;
|
|
|
|
/**
|
|
* Optional label for the emission, could be used for documentation or UI.
|
|
*/
|
|
label?: string;
|
|
|
|
/**
|
|
* This is purely for documentation purposes,
|
|
* it doesn't affect the execution of the step.
|
|
*
|
|
* In Workbench, it will render differently based on this value.
|
|
*/
|
|
conditional?: boolean;
|
|
}
|
|
|
|
|
|
export type CronConfig = {
|
|
/**
|
|
* Should always be cron
|
|
*/
|
|
type: 'cron'
|
|
|
|
/**
|
|
* A unique name for this cron step, used internally and for linking handlers.
|
|
*/
|
|
name: string
|
|
|
|
/**
|
|
* Optional human-readable description.
|
|
*/
|
|
description?: string
|
|
|
|
/**
|
|
* The cron expression for scheduling.
|
|
*/
|
|
cron: string
|
|
|
|
/**
|
|
* Optional: Topics that are virtually emitted, perhaps for documentation or lineage, but not strictly required for execution.
|
|
*/
|
|
virtualEmits?: Emit[]
|
|
|
|
/**
|
|
* Topics this cron step can emit events to.
|
|
*/
|
|
emits: Emit[]
|
|
|
|
/**
|
|
* Optional: An array of flow names this step belongs to.
|
|
*/
|
|
flows?: string[]
|
|
|
|
/**
|
|
* Files to include in the step bundle.
|
|
* Needs to be relative to the step file.
|
|
*/
|
|
includeFiles?: string[]
|
|
}
|
|
```
|
|
|
|
### Handler
|
|
|
|
The handler is a function that is exported via `export const handler` that is a `CronHandler` type.
|
|
|
|
**TypeScript/JavaScript:**
|
|
```typescript
|
|
/**
|
|
* CRON handler accepts only one argument, the FlowContext.
|
|
*
|
|
* The FlowContext is based on the Handlers which can vary depending on the config['emits'].
|
|
*/
|
|
export const handler: Handlers['CronJobEvery5Minutes'] = async ({ logger, emit, traceId, state, streams }) => {
|
|
logger.info('CRON Job Every 5 Minutes started')
|
|
}
|
|
```
|
|
|
|
**Python:**
|
|
```python
|
|
async def handler(context):
|
|
# context: object containing emit, logger, state, streams, trace_id
|
|
context.logger.info("CRON Job Every 5 Minutes started")
|
|
```
|
|
|
|
### Examples of Cron expressions
|
|
|
|
- `0 0 * * *`: Runs daily at midnight
|
|
- `*/5 * * * *`: Runs every 5 minutes
|
|
- `0 9 * * *`: Runs daily at 9 AM
|
|
- `0 9 * * 1`: Runs every Monday at 9 AM
|
|
- `0 9 * * 1-5`: Runs every Monday to Friday at 9 AM
|
|
|
|
### Example uses of Cron Steps
|
|
|
|
- Sending email notifications on a regular basis
|
|
- Cleaning up old records
|
|
- Purging old data
|
|
- Generating reports
|
|
- Sending out scheduled notifications
|
|
- Collecting metrics from third-party services
|
|
- Reconciling data from different sources
|
|
|
|
## Examples
|
|
|
|
### TypeScript Example
|
|
|
|
```typescript
|
|
export const config: CronConfig = {
|
|
type: 'cron',
|
|
name: 'CronJobEvery5Minutes', // should always be the same as Handlers['__']
|
|
cron: '*/5 * * * *',
|
|
emits: [], // No emits in this example
|
|
flows: ['example-flow']
|
|
};
|
|
|
|
export const handler: Handlers['CronJobEvery5Minutes'] = async ({ logger }) => {
|
|
logger.info('Cron job started')
|
|
}
|
|
```
|
|
|
|
### Python Example
|
|
|
|
```python
|
|
config = {
|
|
"type": "cron",
|
|
"name": "CronJobEvery5Minutes",
|
|
"cron": "*/5 * * * *", # Run every 5 minutes
|
|
"emits": [], # No emits in this example
|
|
"flows": ["example-flow"]
|
|
}
|
|
|
|
async def handler(context):
|
|
context.logger.info("Cron job started")
|
|
``` |