Nixtla Production Pipeline Generator

Transform validated forecasting experiments into production-ready inference pipelines with proper orchestration, monitoring, and error handling.

Overview

This skill productionizes Nixtla forecasting workflows by generating complete deployment artifacts:

Airflow DAGs: Enterprise orchestration with dependencies and monitoring
Prefect Flows: Modern Python-native pipelines with better local testing
Cron Scripts: Simple single-machine batch processing

All pipelines implement: Extract -> Transform -> Forecast -> Load -> Monitor

Prerequisites

Required:

Python 3.8+
Completed experiment in forecasting/config.yml
One of: Airflow, Prefect, or cron access

Environment Variables:

NIXTLA_API_KEY: TimeGPT API key (if using TimeGPT)
FORECAST_DATA_SOURCE: Production data connection string
FORECAST_DESTINATION: Output destination for forecasts

Installation:

pip install nixtla pandas statsforecast  # Core
pip install apache-airflow  # For Airflow
pip install prefect  # For Prefect

Instructions

Step 1: Read Experiment Config

Load experiment from forecasting/config.yml:

python {baseDir}/scripts/read_experiment.py --config forecasting/config.yml

Step 2: Select Orchestration Platform

Choose based on requirements:

Airflow: Enterprise, complex dependencies, extensive monitoring
Prefect: Python-native, better local testing, modern error handling
Cron: Simple single-machine, no dependencies, quick setup

Step 3: Generate Pipeline

python {baseDir}/scripts/generate_pipeline.py \
    --config forecasting/config.yml \
    --platform airflow \
    --output pipelines/

Step 4: Add Monitoring

python {baseDir}/scripts/add_monitoring.py \
    --pipeline pipelines/forecast_dag.py \
    --metrics smape,mase

Step 5: Deploy

Follow generated pipelines/README.md for deployment instructions.

Output

pipelines/forecast_dag.py: Main pipeline file (Airflow/Prefect/Cron)
pipelines/monitoring.py: Quality checks and fallback logic
pipelines/README.md: Deployment instructions
pipelines/requirements.txt: Dependencies

Error Handling

Error: Config file not found Solution: Run nixtla-experiment-architect first to create config
Error: NIXTLA_API_KEY not set Solution: Export your TimeGPT API key or use StatsForecast baselines
Error: Database connection failed Solution: Verify FORECAST_DATA_SOURCE connection string
Error: Forecast quality check failed Solution: Pipeline auto-falls back to baseline models

Examples

Example 1: Airflow DAG

python {baseDir}/scripts/generate_pipeline.py \
    --config forecasting/config.yml \
    --platform airflow \
    --schedule "0 6 * * *" \
    --output pipelines/

Output:

Generated: pipelines/forecast_dag.py
Schedule: Daily at 6am
Tasks: extract -> transform -> forecast -> load -> monitor

Example 2: Simple Cron Script

python {baseDir}/scripts/generate_pipeline.py \
    --config forecasting/config.yml \
    --platform cron \
    --output pipelines/

Resources

Scripts: {baseDir}/scripts/
Templates: {baseDir}/assets/templates/
Nixtla Docs: https://nixtla.github.io/

Related Skills:

nixtla-experiment-architect: Creates experiments to productionize
nixtla-timegpt-finetune-lab: Fine-tuned models for pipelines
nixtla-usage-optimizer: Cost-effective routing strategies

nixtla-prod-pipeline-generator