Automating batch processing

Automate batch processing by scheduling Flows or Agents to run against Records on a recurring basis.

Use cases

• Nightly data enrichment: Process new Records added during the day
• Weekly reports: Summarize activity across all Records
• Periodic cleanup: Archive or update old Records
• Scheduled updates: Refresh cached data from external sources

Set up automated batch processing

1. Click Schedules in the sidebar
2. Click Create schedule
3. Configure:
   • Flow or Agent: Select your processing Flow or Agent
   • Target: Choose Record Type, specific Record IDs, or a Record Filter
   • Timing: Set a recurring schedule (for example, daily at 2 AM)
4. Click Create

Targeting Records

By Record Type

Process every Record of a given type each time the Schedule runs. New Records added to that type are included automatically.

By Record IDs

Process a fixed set of Records by specifying their IDs.

By Record Filter

Process only Records matching filter conditions on metadata fields. Filters use a structured DSL with operators such as eq, contains, gt, withinLastDays, and others.

Example filter targeting unprocessed Records:

1
{
2
  "type": "customers",
3
  "where": {
4
    "op": "and",
5
    "conditions": [
6
      { "field": "processed", "op": "eq", "value": false }
7
    ]
8
  }
9
}

Marking Records as processed

Update Records after processing to avoid reprocessing:

1. In your Flow, add an Upsert Record step at the end
2. Update the Record’s metadata to include a processed flag
3. Use a Record Filter on your Schedule to skip already-processed Records on subsequent runs

Monitoring batch Schedules

1. Go to Schedules
2. Find your batch Schedule
3. View Run History below the Schedules list
4. See when it last ran, its status, and progress

Handling failures

If some Records fail during batch processing:

1. Check execution logs for specific errors
2. Failed Records remain unprocessed if you use a filter that skips processed Records
3. The next scheduled run retries them
4. If failures persist, update your Flow to handle edge cases

Cost optimization

Batch processing can be expensive. Optimize costs by:

• Processing only necessary Records using Record Filters
• Running less frequently if daily is not required
• Using cheaper AI models for batch jobs

Next steps

• What are Schedules? for basic Schedule setup
• What are Records? to understand Record structure
• What are Logs? to troubleshoot scheduled runs