Building Internal Developer Platforms with Windmill: Automating CI/CD Hooks and Database Migrations

Internal Developer Platforms (IDPs) are transforming how modern engineering teams operate. By providing a self-service layer on top of complex infrastructure, IDPs empower developers to deploy code and manage data changes without needing deep expertise in underlying DevOps tools. Windmill has emerged as a powerful contender in this space, offering a no-code/low-code workflow automation engine that can orchestrate complex backend tasks with ease. In this post, we explore how to leverage Windmill to automate critical aspects of the software delivery lifecycle: CI/CD hooks and database migrations.

Why Windmill for Your IDP?

Traditional CI/CD pipelines are often brittle, requiring significant YAML configuration and external tooling. Windmill simplifies this by allowing you to define workflows as code (TypeScript, Python, Go, etc.) or even as visual scripts. Its key advantages for an IDP include:

• Git-Native Workflows: Your automation scripts live in your repository, version-controlled and reviewable.
• Real-Time API Exposure: Every Windmill script can be exposed as a REST API endpoint instantly.
• Secrets Management: Built-in integration with HashiCorp Vault or environment variables for secure credential handling.
• Observability: Detailed logs and execution history for every workflow run.

Automating CI/CD Hooks with Windmill

One of the most common use cases is reacting to Git events. When a developer pushes to the main branch, you might want to trigger a deployment, update documentation, or notify stakeholders. Instead of relying solely on GitHub Actions or GitLab CI, you can use Windmill as a lightweight, customizable webhook handler.

Example: Triggering a Deployment

Suppose you have a simple deployment script that runs kubectl apply to your Kubernetes cluster. You can wrap this in a Windmill script and expose it as a webhook.

import { request } from "windmill-client";

async function main(webhook_payload: any) {
  const branch = webhook_payload.ref;
  if (branch !== "refs/heads/main") {
    return { status: "ignored", reason: "Not main branch" };
  }

  // Simulate a deployment step
  console.log("Deploying to production...");
  
  // In a real scenario, you would call your deployment API here
  // Example: await request.post("https://deploy.yourcompany.com/trigger", { data: { service: "api-gateway" } });

  return { status: "success", message: "Deployment triggered for main branch" };
}

You then configure your Git provider to send a POST request to the URL generated by Windmill for this script. This creates a flexible, auditable hook that can be extended with complex logic without touching your main CI/CD configuration.

Safe and Automated Database Migrations

Database migrations are a critical but risky part of the deployment process. Mistakes here can lead to data loss or downtime. Windmill allows you to encapsulate migration steps into secure, idempotent workflows that can be triggered manually from an IDP dashboard or automatically via API.

Example: Running Migrations with Error Handling

Here’s an example of a Python script in Windmill that runs a migration, captures the output, and sends an alert on failure.

import subprocess
import requests
import json

def main(env: str) -> str:
    # Determine the migration command based on environment
    command = ["python", "manage.py", "migrate", "--database", env]
    
    try:
        # Run the migration
        result = subprocess.run(command, capture_output=True, text=True, check=True)
        print(f"Migration successful:\n{result.stdout}")
        
        # Send success notification
        requests.post("https://hooks.slack.com/services/xxx/yyy/zzz", 
                      json={"text": f"Migration successful for {env}"}).raise_for_status()
        return "Migration completed successfully"
    
    except subprocess.CalledProcessError as e:
        print(f"Migration failed: {e.stderr}")
        # Send failure alert
        requests.post("https://hooks.slack.com/services/xxx/yyy/zzx", 
                      json={"text": f"Migration failed for {env}: {e.stderr}"}).raise_for_status()
        raise e

This script can be exposed as an API endpoint /api/w/v1/run/migrate-env. Your IDP can then provide a simple UI where developers select an environment and trigger the migration, ensuring that all changes are logged and alerts are sent automatically.

Best Practices for IDP Integration

1. Idempotency: Ensure your Windmill scripts are idempotent, especially for database operations, to prevent issues on retries.
2. Access Control: Use Windmill’s role-based access control to restrict who can trigger sensitive workflows like production migrations.
3. Logging: Always log detailed output. Windmill’s built-in logging helps in debugging failed workflows quickly.
4. Testing: Develop and test your workflows in a dev branch before merging to main. Windmill allows you to run scripts locally and in staging environments.

Conclusion

Building an internal developer platform doesn’t require reinventing the wheel. Tools like Windmill provide a robust, flexible foundation for automating the most tedious and error-prone parts of the development lifecycle. By automating CI/CD hooks and database migrations, you can reduce deployment friction, improve reliability, and empower your developers to focus on what they do best: building great software.

Ready to get started? Try installing Windmill in your local environment and exposing your first script as an API. Your IDP journey starts with a single workflow.