Skip to main content

AWSNodeClass API Reference

The AWSNodeClass resource is a cluster-scoped custom resource that defines AWS EC2 configuration for nodes managed by Stratos NodePools. This resource follows the Karpenter pattern of separating cloud-specific configuration from pool management, enabling reusability and independent schema evolution per cloud provider.

Overview

AWSNodeClass decouples AWS-specific instance configuration from NodePool resources:

  • NodePool: Defines pool sizing, scaling behavior, and node template (labels, taints)
  • AWSNodeClass: Defines AWS EC2 configuration (instance type, AMI, networking, IAM)

This separation allows:

  • Multiple NodePools to share the same EC2 configuration
  • Independent updates to pool sizing vs. instance configuration
  • Clean path to multi-cloud support (GCPNodeClass, AzureNodeClass)

Resource Definition

apiVersion: stratos.sh/v1alpha1
kind: AWSNodeClass
metadata:
name: <nodeclass-name>
spec:
# Bootstrap template for userData generation (required)
bootstrapTemplate: <string> # AL2023, AL2, or Bottlerocket
region: <string>
instanceType: <string>
architecture: <string>
# AMI: use one of ami or amiSelector
ami: <string>
amiSelector: <AMISelector>
# Subnets: use one of subnetIds or subnetSelector
subnetIds: <[]string>
subnetSelector: <SubnetSelector>
# Security groups: use one of securityGroupIds or securityGroupSelector
securityGroupIds: <[]string>
securityGroupSelector: <SecurityGroupSelector>
# IAM: use one of iamInstanceProfile or role
iamInstanceProfile: <string>
role: <string>
# IMDS configuration
metadataOptions: <MetadataOptions>
# Custom scripts to merge with generated bootstrap (optional)
customUserData: <string>
blockDeviceMappings: <[]BlockDeviceMapping>
tags: <map[string]string>
spotConfig:
instanceTypes: <[]string>
allocationStrategy: <string>
maxPrice: <string>
status:
nodePoolCount: <int32>
resolvedAMI: <string>
resolvedSubnets: <[]ResolvedSubnet>
resolvedSecurityGroups: <[]ResolvedSecurityGroup>
resolvedInstanceProfile: <string>
resolvedLaunchTemplateID: <string>
conditions: <[]Condition>

Spec Fields

Required Fields

FieldTypeDescription
bootstrapTemplatestringBootstrap template for userData generation: AL2023, AL2, or Bottlerocket. Stratos generates the complete bootstrap script using cluster configuration from Helm values.
instanceTypestringEC2 instance type (e.g., m5.large, c5.xlarge, m8g.large).

Bootstrap Templates

TemplateAMI FamilyBootstrap FormatWarmup Mode
AL2023Amazon Linux 2023nodeadm MIME multipartSelfStop
AL2Amazon Linux 2bootstrap.sh MIME multipartSelfStop
BottlerocketBottlerocketTOML configurationControllerStop

AMI Configuration (one required)

Exactly one of ami or amiSelector must be specified. Setting both is rejected by CEL validation.

FieldTypeDescription
amistringStatic AMI ID. Must match pattern ^ami-[a-z0-9]+$.
amiSelectorAMISelectorDynamic AMI selection by tags, name, and/or owner. The newest matching AMI is selected.

Subnet Configuration (one required)

Exactly one of subnetIds or subnetSelector must be specified.

FieldTypeDescription
subnetIds[]stringStatic list of subnet IDs. Minimum 1. Instances are distributed round-robin.
subnetSelectorSubnetSelectorDynamic subnet selection by tags. All matching subnets are used for round-robin placement.

Security Group Configuration (one required)

Exactly one of securityGroupIds or securityGroupSelector must be specified.

FieldTypeDescription
securityGroupIds[]stringStatic list of security group IDs. Minimum 1.
securityGroupSelectorSecurityGroupSelectorDynamic security group selection by tags and/or name.

IAM Configuration (one required)

Exactly one of iamInstanceProfile or role must be specified.

FieldTypeDescription
iamInstanceProfilestringStatic IAM instance profile ARN or name.
rolestringIAM role name. The controller creates and manages an instance profile named stratos-<cluster>-<name> automatically.

Optional Fields

FieldTypeDefaultDescription
regionstringController regionAWS region for launching instances (e.g., us-east-1).
architecturestringx86_64Instance architecture: x86_64 or arm64. Used for AMI selection when using amiSelector.
metadataOptionsMetadataOptionsEC2 defaultsIMDS configuration for launched instances.
customUserDatastring-Additional scripts/config to merge with generated bootstrap. For AL2/AL2023, shell scripts. For Bottlerocket, additional TOML settings.
blockDeviceMappings[]BlockDeviceMapping-EBS volume configuration.
tagsmap[string]string-Additional tags to apply to instances. Stratos management tags are added automatically.

Spot Configuration

Configure Spot instance fleet parameters for use with NodePool spotReplacement. When spotConfig is set, Stratos creates an EC2 Launch Template and uses the EC2 CreateFleet API (type instant, not the legacy SpotFleet API) for diversified Spot instance launches. The fleet builds a cross-product of instanceTypes and resolved subnets as overrides, maximizing availability across types and availability zones.

FieldTypeRequiredDefaultDescription
spotConfig.instanceTypes[]stringYes (if spotConfig set)-Instance types for Spot fleet diversification. Multiple types increase the chance of getting capacity. Minimum 1.
spotConfig.allocationStrategystringNoprice-capacity-optimizedSpot fleet allocation strategy.
spotConfig.maxPricestringNoOn-Demand priceMaximum Spot price. Empty string uses On-Demand price as cap.
Example
spotConfig:
instanceTypes:
- m5.large
- m5a.large
- m5d.large
allocationStrategy: price-capacity-optimized
maxPrice: "0.05"

AMISelector

Selects an AMI dynamically. All specified fields use AND semantics. When multiple AMIs match, the most recently created one is selected.

FieldTypeRequiredDescription
tagsmap[string]stringNoTag key-value pairs to match (AND semantics).
namestringNoAMI name pattern. Supports wildcards (e.g., my-eks-ami-*).
ownerstringNoOwner account ID or alias (self, amazon).
Example
amiSelector:
name: "my-eks-ami-*"
owner: self
tags:
kubernetes.io/os: linux

SubnetSelector

Selects subnets dynamically by tags.

FieldTypeRequiredDescription
tagsmap[string]stringNoTag key-value pairs to match (AND semantics).
Example
subnetSelector:
tags:
stratos.sh/discovery: my-cluster

SecurityGroupSelector

Selects security groups dynamically by tags and/or name.

FieldTypeRequiredDescription
tagsmap[string]stringNoTag key-value pairs to match (AND semantics).
namestringNoSecurity group name pattern. Supports wildcards (e.g., stratos-nodes-*).
Example
securityGroupSelector:
tags:
stratos.sh/discovery: my-cluster
name: "stratos-*"

MetadataOptions

Controls the Instance Metadata Service (IMDS) configuration for launched instances. Maps directly to EC2 InstanceMetadataOptionsRequest.

FieldTypeRequiredValidationDescription
httpTokensstringNorequired or optionalrequired enforces IMDSv2; optional allows both v1 and v2.
httpPutResponseHopLimit*int32No1--64HTTP PUT response hop limit for IMDS token requests. Set to 2 for containerized workloads.
httpEndpointstringNoenabled or disabledControls whether the IMDS endpoint is available.
Example: Enforce IMDSv2
metadataOptions:
httpTokens: required
httpPutResponseHopLimit: 2
httpEndpoint: enabled

Block Device Mapping

FieldTypeRequiredDefaultDescription
deviceNamestringYes-Device name (e.g., /dev/xvda, /dev/xvdb).
volumeSizeint32Yes-Volume size in GiB. Minimum: 1.
volumeTypestringYes-EBS volume type: gp3, gp2, io1, io2.
encryptedboolNofalseEnable EBS encryption.
iopsint32No-IOPS for io1/io2 volumes, or provisioned IOPS for gp3.
throughputint32No-Throughput in MB/s for gp3 volumes.

Status Fields

The AWSNodeClass status is updated by the controller. When selectors are used, the resolved values are populated after successful resolution.

FieldTypeDescription
nodePoolCountint32Number of NodePools currently referencing this AWSNodeClass.
resolvedAMIstringThe resolved AMI ID (from selector or static field).
resolvedSubnets[]ResolvedSubnetResolved subnets with IDs and availability zones.
resolvedSecurityGroups[]ResolvedSecurityGroupResolved security groups with IDs and names.
resolvedInstanceProfilestringThe resolved instance profile ARN.
resolvedLaunchTemplateIDstringThe EC2 Launch Template ID created for Spot fleet. Populated when spotConfig is set.
conditions[]ConditionCurrent conditions (see below).

ResolvedSubnet

FieldTypeDescription
idstringSubnet ID (e.g., subnet-12345678).
zonestringAvailability zone (e.g., us-east-1a). Empty for static subnet IDs.

ResolvedSecurityGroup

FieldTypeDescription
idstringSecurity group ID (e.g., sg-12345678).
namestringSecurity group name. Empty for static security group IDs.

Conditions

ConditionDescription
ValidThe AWSNodeClass spec is valid.
InUseThe AWSNodeClass is referenced by one or more NodePools.
AMIReadyThe AMI has been resolved successfully.
SubnetsReadySubnets have been resolved successfully.
SecurityGroupsReadySecurity groups have been resolved successfully.
InstanceProfileReadyThe instance profile is ready (static or controller-managed).

Condition Reasons

ReasonApplies ToDescription
ResolvedAMI, Subnets, SGs, InstanceProfileResource successfully resolved.
ResolutionFailedAMI, Subnets, SGs, InstanceProfileAWS API call failed.
NoMatchingResourcesAMI, Subnets, SGsNo resources matched the selector.
RoleNotFoundInstanceProfileThe IAM role does not exist.
RoleUpdateFailedInstanceProfileFailed to update the role on the instance profile.
note

The NodePool controller checks all four readiness conditions before launching instances. If any condition is False, the NodePool is set to Ready=False with reason NodeClassNotReady.

Validation Rules

The CRD uses CEL (Common Expression Language) validation to enforce mutual exclusivity:

RuleError Message
ami or amiSelector must be set"one of ami or amiSelector is required"
ami and amiSelector cannot both be set"ami and amiSelector are mutually exclusive"
subnetIds or subnetSelector must be set"one of subnetIds or subnetSelector is required"
subnetIds and subnetSelector cannot both be set"subnetIds and subnetSelector are mutually exclusive"
securityGroupIds or securityGroupSelector must be set"one of securityGroupIds or securityGroupSelector is required"
securityGroupIds and securityGroupSelector cannot both be set"securityGroupIds and securityGroupSelector are mutually exclusive"
iamInstanceProfile or role must be set"one of iamInstanceProfile or role is required"
iamInstanceProfile and role cannot both be set"role and iamInstanceProfile are mutually exclusive"

These validations run at admission time, providing immediate feedback when applying manifests.

Lifecycle Management

Deletion Protection

AWSNodeClass uses a finalizer (stratos.sh/in-use) to prevent deletion while referenced by NodePools. You must delete all referencing NodePools before deleting an AWSNodeClass.

# This will block if NodePools reference it
kubectl delete awsnodeclass my-nodeclass

# Check which NodePools reference it
kubectl get nodepools -o json | jq '.items[] | select(.spec.template.nodeClassRef.name == "my-nodeclass") | .metadata.name'

# Delete the NodePools first, then the AWSNodeClass
kubectl delete nodepool my-pool
kubectl delete awsnodeclass my-nodeclass

Instance Profile Cleanup

When using spec.role, the controller adds a second finalizer (stratos.sh/instance-profile). On deletion, the controller:

  1. Waits for the stratos.sh/in-use finalizer to be removed (all NodePools deleted)
  2. Removes the role from the instance profile
  3. Deletes the instance profile
  4. Removes the stratos.sh/instance-profile finalizer

This ordering ensures no NodePools are launching with the profile while it is being deleted.

Reference from NodePool

NodePools reference AWSNodeClass via spec.template.nodeClassRef:

apiVersion: stratos.sh/v1alpha1
kind: NodePool
metadata:
name: workers
spec:
poolSize: 10
minStandby: 3
template:
nodeClassRef:
kind: AWSNodeClass
name: standard-nodes
labels:
stratos.sh/pool: workers

Examples

Static IDs (Original Pattern)

awsnodeclass-static.yaml
apiVersion: stratos.sh/v1alpha1
kind: AWSNodeClass
metadata:
name: static-nodes
spec:
bootstrapTemplate: AL2023
instanceType: m5.large
ami: ami-0123456789abcdef0
subnetIds:
- subnet-12345678
- subnet-87654321
securityGroupIds:
- sg-12345678
iamInstanceProfile: arn:aws:iam::123456789012:instance-profile/eks-node-role
awsnodeclass-selectors.yaml
apiVersion: stratos.sh/v1alpha1
kind: AWSNodeClass
metadata:
name: dynamic-nodes
spec:
bootstrapTemplate: AL2023
instanceType: m5.large

amiSelector:
name: "my-eks-ami-*"
owner: self
tags:
kubernetes.io/os: linux

subnetSelector:
tags:
stratos.sh/discovery: my-cluster

securityGroupSelector:
tags:
stratos.sh/discovery: my-cluster

role: my-eks-node-role

metadataOptions:
httpTokens: required
httpPutResponseHopLimit: 2

blockDeviceMappings:
- deviceName: /dev/xvda
volumeSize: 20
volumeType: gp3
encrypted: true

Mixed (Static AMI, Dynamic Networking)

awsnodeclass-mixed.yaml
apiVersion: stratos.sh/v1alpha1
kind: AWSNodeClass
metadata:
name: mixed-nodes
spec:
bootstrapTemplate: AL2023
instanceType: m5.large
ami: ami-0123456789abcdef0

subnetSelector:
tags:
stratos.sh/discovery: my-cluster

securityGroupSelector:
tags:
stratos.sh/discovery: my-cluster

iamInstanceProfile: arn:aws:iam::123456789012:instance-profile/eks-node-role

Bottlerocket with Selectors

awsnodeclass-bottlerocket-selectors.yaml
apiVersion: stratos.sh/v1alpha1
kind: AWSNodeClass
metadata:
name: bottlerocket-dynamic
spec:
# Stratos generates Bottlerocket TOML configuration automatically
bootstrapTemplate: Bottlerocket
instanceType: m5.large
architecture: x86_64

amiSelector:
name: "bottlerocket-aws-k8s-*-x86_64-*"
owner: amazon

subnetSelector:
tags:
stratos.sh/discovery: my-cluster

securityGroupSelector:
tags:
stratos.sh/discovery: my-cluster

role: my-eks-node-role

metadataOptions:
httpTokens: required
httpPutResponseHopLimit: 2

blockDeviceMappings:
- deviceName: /dev/xvda
volumeSize: 8
volumeType: gp3
encrypted: true
- deviceName: /dev/xvdb
volumeSize: 20
volumeType: gp3
encrypted: true

# Optional: Additional Bottlerocket settings
customUserData: |
[settings.host-containers.admin]
enabled = true

Spot-Enabled Configuration

awsnodeclass-spot.yaml
apiVersion: stratos.sh/v1alpha1
kind: AWSNodeClass
metadata:
name: spot-enabled
spec:
bootstrapTemplate: AL2023
instanceType: m5.large

subnetSelector:
tags:
stratos.sh/discovery: my-cluster

securityGroupSelector:
tags:
stratos.sh/discovery: my-cluster

role: my-eks-node-role

blockDeviceMappings:
- deviceName: /dev/xvda
volumeSize: 50
volumeType: gp3
encrypted: true

spotConfig:
instanceTypes:
- m5.large
- m5a.large
- m5d.large
- m5ad.large
allocationStrategy: price-capacity-optimized

The instanceType field defines the On-Demand instance type. The spotConfig.instanceTypes list defines the Spot fleet diversification pool. Using multiple instance types increases the chance of getting Spot capacity.

Finding AMI IDs

EKS-Optimized AMIs

# Amazon Linux 2 (x86_64)
aws ssm get-parameter \
--name /aws/service/eks/optimized-ami/1.34/amazon-linux-2/recommended/image_id \
--query "Parameter.Value" --output text

# Amazon Linux 2023 (x86_64)
aws ssm get-parameter \
--name /aws/service/eks/optimized-ami/1.34/amazon-linux-2023/x86_64/standard/recommended/image_id \
--query "Parameter.Value" --output text

# Amazon Linux 2023 (ARM64)
aws ssm get-parameter \
--name /aws/service/eks/optimized-ami/1.34/amazon-linux-2023/arm64/standard/recommended/image_id \
--query "Parameter.Value" --output text

# Bottlerocket (x86_64)
aws ssm get-parameter \
--name /aws/service/bottlerocket/aws-k8s-1.34/x86_64/latest/image_id \
--query "Parameter.Value" --output text

# Bottlerocket (ARM64)
aws ssm get-parameter \
--name /aws/service/bottlerocket/aws-k8s-1.34/arm64/latest/image_id \
--query "Parameter.Value" --output text

Replace 1.34 with your EKS cluster version.

Kubectl Commands

# List all AWSNodeClasses
kubectl get awsnodeclasses

# Short name
kubectl get awsnc

# Get detailed status including resolved resources
kubectl describe awsnodeclass dynamic-nodes

# Get YAML output to see resolved status
kubectl get awsnodeclass dynamic-nodes -o yaml

# Check which NodePools reference an AWSNodeClass
kubectl get nodepools -o custom-columns='NAME:.metadata.name,NODECLASS:.spec.template.nodeClassRef.name'

# Check resolved AMI
kubectl get awsnodeclass dynamic-nodes -o jsonpath='{.status.resolvedAMI}'

# Check all conditions
kubectl get awsnodeclass dynamic-nodes -o jsonpath='{.status.conditions[*].type}={.status.conditions[*].status}'

Next Steps