Pipeline Orchestration

What is Kestra, and How to Deploy It in an Enterprise Data Stack?

Last updated on
May 12, 2026

What is Kestra?

Kestra is an open-source orchestrator designed for both scheduled and event-driven workflows. It stands out by integrating Infrastructure as Code practices into data, process, and microservice orchestration, allowing for the creation of dependable workflows with just a few lines of code. The platform's declarative YAML interface and versatile UI make it accessible to both developers and business professionals, fostering collaborative workflow creation. With Kestra, users benefit from simplified workflow management, the ability to adapt quickly to changes through UI or API, and a robust set of developer tools. This unique combination ensures users can efficiently build and manage complex workflows, making Kestra a superior choice for reliable process orchestration.

Watch Kestra in action

No items found.

Why is Kestra better on Shakudo?

Getting Started & Usage

Kestra is used inside the Shakudo SaaS platform. Customers do not install Kestra or manage its infrastructure manually. Shakudo handles deployment, authentication, routing, and platform access; the customer journey starts from the Shakudo UI and then continues inside Kestra.

<aside>📌 Standard stack-component usage pattern: log into Shakudo → open the stack component → use the component UI → validate the first workflow.

</aside>

Step 1 — Access Kestra in Shakudo

Start from the Shakudo platform, not from a standalone Kestra URL or local deployment.

  • Log in to the Shakudo SaaS platform using your customer account.
  • Open the main Shakudo dashboard for your environment.

Step 2 — Open Kestra UI

Kestra is available as a stack component inside Shakudo. Use the stack component entry point so access, routing, and permissions are handled by the platform.

  • From the Shakudo dashboard, go to the Stack Components or Applications area.
  • Find and select Kestra.
  • Click Open, Launch, or the available Kestra entry action.
  • Kestra opens in the browser through Shakudo-managed access.
  • If the page does not open, check whether the Kestra stack component is enabled for your environment and whether your user has permission to access it.

Step 3 — Create Your First Workflow

After Kestra opens, create a small workflow first. The goal is to confirm that the UI, permissions, execution engine, and logs are working before building production workflows.

  • In Kestra, open the Flows page.
  • Click New Flow.
  • Choose a namespace such as dev, sandbox, or your team namespace.
  • Paste a simple test flow and save it.

id: my-first-flow
namespace: dev

tasks:
 - id: hello
   type: io.kestra.plugin.core.log.Log
   message: "Hello from Kestra inside Shakudo"

Use a non-production namespace for the first test. This keeps onboarding safe and avoids accidentally triggering customer workloads.

Step 4 — Configure and Run

Run the workflow manually first, then review the execution details. This is the fastest way to confirm the customer can operate Kestra through Shakudo.

  • Click Execute or Run from the flow page.
  • Open the execution that was created.
  • Confirm the status changes from Running to Success.
  • Open the Logs tab and confirm the hello message is visible.
  • If the execution fails, copy the task error and share it with the Shakudo support team or your platform administrator.

Step 5 — Understand the Basic Kestra Concepts

Once the first run succeeds, use these concepts to build real workflows.

  • Flow: the YAML definition of a workflow.
  • Namespace: a logical folder for flows, usually separated by team, environment, or use case.
  • Task: one step inside a workflow, such as calling an API, running a script, moving a file, or writing to a database.
  • Execution: one run of a flow, including status, logs, inputs, outputs, and timing.
  • Trigger: an optional rule that starts a flow automatically, such as a schedule, webhook, or another flow completing.

Step 6 — Build a Practical Workflow

After the first test flow works, create a workflow that matches a real customer use case. Start small and validate each step before adding more automation.

  • For a scheduled workflow, add a schedule trigger and confirm the timezone and frequency.
  • For an event-driven workflow, use a webhook trigger and test it with a sample payload.
  • For a data workflow, read from a known source first, then write to a safe destination.
  • For an AI or automation workflow, keep prompts, API keys, and external service credentials managed through the customer-approved secret process.

id: daily-health-check
namespace: dev

triggers:
 - id: daily
   type: io.kestra.plugin.core.trigger.Schedule
   cron: "0 9 * * *"

tasks:
 - id: log-start
   type: io.kestra.plugin.core.log.Log
   message: "Daily workflow started from Shakudo-managed Kestra"

Step 7 — Monitor Executions

Use the Kestra UI to confirm that workflows are running correctly before relying on them operationally.

  • Open Executions to see current and historical runs.
  • Use Status to identify Success, Failed, Running, or Killed executions.
  • Open Logs to debug task-level issues.
  • Use Outputs to inspect values or files produced by tasks.
  • For production workflows, agree on who owns monitoring and who should be notified when a workflow fails.

Step 8 — Promote to Production Usage

Only move a flow toward production after the customer has validated access, execution, logs, and ownership.

  • Move the flow from a dev/sandbox namespace to the agreed production namespace.
  • Confirm required credentials and external connections are approved for production.
  • Add retries or failure handling for tasks that call external systems.
  • Document the workflow owner, schedule, expected output, and escalation path.
  • Run one controlled production test before enabling scheduled or event-driven automation.

Useful UI Areas

  • Shakudo Dashboard: starting point for customer access to Kestra.
  • Stack Components / Applications: where the customer opens Kestra from Shakudo.
  • Kestra Flows: create, edit, and version workflows.
  • Kestra Executions: monitor runs, logs, task state, and outputs.
  • Kestra Triggers: review schedules, webhooks, and event-based starts.
  • Kestra Plugins: browse available task types when building workflows.

Why is better on Shakudo?

Core Shakudo Features

Own Your AI

Keep data sovereign, protect IP, and avoid vendor lock-in with infra-agnostic deployments.

Faster Time-to-Value

Pre-built templates and automated DevOps accelerate time-to-value.
integrate

Flexible with Experts

Operating system and dedicated support ensure seamless adoption of the latest and greatest tools.
See Shakudo in Action
Neal Gilmore
Get Started >