AWS / ECS Deployment

Causet’s production deployment targets AWS ECS (Elastic Container Service). Each service runs as a separate ECS service backed by Fargate tasks. Infrastructure is managed with Terraform modules in the infra/ directory.

AWS services used

AWS Service	Causet use
ECS (Fargate)	Run all Causet service containers
ECR	Store Docker images
RDS PostgreSQL	Two databases: `causet` (event store) + `projections` (read models)
MSK (or self-managed Redpanda)	Kafka-compatible broker
ElastiCache Redis	IR artifact cache + optional query result cache
S3	IR artifact storage per release version
ALB	HTTP routing to runtime (8080) and query service (8082)
Secrets Manager	DB credentials, Clerk secret key, API keys
IAM	ECS task roles for S3 and Secrets Manager access
VPC	Network isolation: private subnets for services, public subnet for ALB

Architecture

ECS service definitions

Each Causet service maps to one ECS service. Recommended Fargate CPU/memory:

ECS Service	CPU	Memory
`causet-runtime-service`	1024 (1 vCPU)	2048 MB
`causet-projection-worker`	1024 (1 vCPU)	2048 MB
`causet-query-service`	512	1024 MB
`causet-saas-cloud`	512	1024 MB
`causet-cloud-control-plane`	256	512 MB
`ws-gateway-go`	256	512 MB

Task definition: causet-runtime-service

{
  "family": "causet-runtime-service",
  "networkMode": "awsvpc",
  "requiresCompatibilities": ["FARGATE"],
  "cpu": "1024",
  "memory": "2048",
  "executionRoleArn": "arn:aws:iam::ACCOUNT:role/ecsTaskExecutionRole",
  "taskRoleArn": "arn:aws:iam::ACCOUNT:role/causet-runtime-task-role",
  "containerDefinitions": [
    {
      "name": "causet-runtime-service",
      "image": "ACCOUNT.dkr.ecr.REGION.amazonaws.com/causet-runtime-service:VERSION",
      "portMappings": [
        { "containerPort": 8080, "protocol": "tcp" },
        { "containerPort": 9090, "protocol": "tcp" }
      ],
      "environment": [
        { "name": "SERVER_PORT", "value": "8080" },
        { "name": "GRPC_PORT", "value": "9090" },
        { "name": "KAFKA_BOOTSTRAP_SERVERS", "value": "broker.kafka.internal:9092" },
        { "name": "S3_BUCKET", "value": "causet-artifacts-prod" },
        { "name": "CAUSET_SAAS_URL", "value": "http://causet-saas-cloud.internal:8085" }
      ],
      "secrets": [
        { "name": "DATABASE_URL", "valueFrom": "arn:aws:secretsmanager:REGION:ACCOUNT:secret:causet/prod/runtime-db-url" },
        { "name": "AWS_ACCESS_KEY_ID", "valueFrom": "arn:aws:secretsmanager:REGION:ACCOUNT:secret:causet/prod/s3-key-id" },
        { "name": "AWS_SECRET_ACCESS_KEY", "valueFrom": "arn:aws:secretsmanager:REGION:ACCOUNT:secret:causet/prod/s3-secret" }
      ],
      "healthCheck": {
        "command": ["CMD-SHELL", "curl -f http://localhost:8080/actuator/health || exit 1"],
        "interval": 30,
        "timeout": 5,
        "retries": 3,
        "startPeriod": 90
      },
      "logConfiguration": {
        "logDriver": "awslogs",
        "options": {
          "awslogs-group": "/ecs/causet-runtime-service",
          "awslogs-region": "us-east-1",
          "awslogs-stream-prefix": "ecs"
        }
      }
    }
  ]
}

See ecs-env-vars-reference.md in the repository for the complete environment variable list for each service.

IAM roles

ECS task execution role

The standard ecsTaskExecutionRole with AmazonECSTaskExecutionRolePolicy attached. This allows ECS to pull images from ECR and write to CloudWatch Logs.

ECS task role (application role)

Each service’s task role needs specific permissions:

causet-runtime-service:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": ["s3:GetObject", "s3:ListBucket"],
      "Resource": [
        "arn:aws:s3:::causet-artifacts-prod",
        "arn:aws:s3:::causet-artifacts-prod/*"
      ]
    },
    {
      "Effect": "Allow",
      "Action": ["secretsmanager:GetSecretValue"],
      "Resource": "arn:aws:secretsmanager:REGION:ACCOUNT:secret:causet/prod/*"
    }
  ]
}

causet-projection-worker additionally needs S3 read access for IR artifacts (same as runtime).

VPC layout

VPC: 10.0.0.0/16
  Public subnets (ALB):
    10.0.1.0/24  us-east-1a
    10.0.2.0/24  us-east-1b
  Private subnets (ECS tasks, RDS, MSK, Redis):
    10.0.10.0/24  us-east-1a
    10.0.11.0/24  us-east-1b

ALB resides in public subnets. ECS tasks have no public IPs.
RDS, MSK, and ElastiCache reside in private subnets alongside ECS tasks.
Use VPC endpoints for S3 to avoid data transfer through the public internet.

ALB configuration

Create two ALB listeners / target groups:

Target group	Port	Health check path
`causet-runtime-tg`	8080	`/actuator/health`
`causet-query-tg`	8082	`/actuator/health`
`causet-saas-tg`	8085	`/actuator/health`

Route by path or subdomain depending on your domain setup. The gRPC port (9090) on the runtime service should be exposed through an NLB (Network Load Balancer) rather than the ALB if gRPC clients connect directly.

Auto-scaling

Scale ECS services based on CPU and memory utilization:

resource "aws_appautoscaling_policy" "runtime_scale_up" {
  name               = "runtime-cpu-scale-up"
  resource_id        = "service/causet-prod/causet-runtime-service"
  scalable_dimension = "ecs:service:DesiredCount"
  service_namespace  = "ecs"
 
  target_tracking_scaling_policy_configuration {
    target_value = 70.0
    predefined_metric_specification {
      predefined_metric_type = "ECSServiceAverageCPUUtilization"
    }
    scale_in_cooldown  = 300
    scale_out_cooldown = 60
  }
}

Minimum 2 tasks for the runtime and projection worker to survive an AZ failure. The projection worker auto-scales based on Kafka consumer lag rather than CPU — use a custom metric.

Deployment strategy

Use ECS rolling updates. Set the deployment configuration:

"deploymentConfiguration": {
  "minimumHealthyPercent": 50,
  "maximumPercent": 200
}

This keeps at least half the current tasks running during a deploy, and allows 2× the desired count during the transition.

Warning: The projection worker is a Kafka consumer. During a rolling update, two versions of the worker may run briefly with the same consumer group. This is safe because the new tasks will rebalance partitions from the old tasks — no duplicate processing. Messages already committed will not be reprocessed.

Deployment checklist

Before deploying to ECS:

New Docker images pushed to ECR with correct version tag
IR artifacts uploaded to S3 at the new irVersion path
Release created in causet-saas-cloud
ECS task definitions updated with the new image tags
Health checks passing on the new image in a test environment

After deploying:

ECS deployment status shows COMPLETED (not stuck)
All tasks show RUNNING and healthy
/actuator/health returns UP on all services
Submit a test intent and verify the response

See Release Checklist for the full procedure.

Terraform

Infrastructure is defined in infra/terraform/. Reference modules:

infra/terraform/
  modules/
    ecs-service/       — reusable ECS service module
    rds/               — RDS PostgreSQL
    msk/               — MSK Kafka
    elasticache/       — Redis
    alb/               — Application Load Balancer
  environments/
    prod/              — production configuration
    staging/           — staging configuration

# Plan and apply
cd infra/terraform/environments/prod
terraform init
terraform plan -out=tfplan
terraform apply tfplan

What is Causet?