Bug: Template not rendered correctly

[cscetbon]

Description:

I found 2 issues in the way template.yaml is rendered. The first one is that if Architecture is defined in Globals.functions and also in a function then the local version gets merged with the global one instead overwriting it and sam complains about having more than one element in the Architectures list

Error: Function ProgramsFunction property Architectures should be a list of length 1

The other issue is that when building a layer even if with CompatibleArchitectures set sam throws a warning saying that it's not set

Building layer 'MyLayer'
WARNING: Layer 'MyLayer' has BuildArchitecture arm64, which is not listed in CompatibleArchitectures

Steps to reproduce:

a. For Architectures list having more than one element, just define Architectures in Globals.function and in ProgramsFunction as well

Use this template.yaml

Globals:
  Function:
    Architectures:
      - arm64

Resources:
  ProgramsFunction:
    Type: AWS::Serverless::Function
    Properties:
      Architectures:
        - arm64
      InlineCode: |
        def handler(event, context):
          print("Hello, world!")
      Handler: index.handler
      Runtime: python3.12

b. For warning about CompatibleArchitectures, use this template.yaml and put some dummy code in functions/programs and layers/my_layer

Resources:
  ProgramsFunction:
    Type: AWS::Serverless::Function
    Properties:
      Architectures:
        - arm64
      CodeUri: functions/programs
      Handler: handler.handler
      Layers:
        - !Ref MyLayer
      Runtime: python3.12

  MyLayer:
    Type: AWS::Serverless::LayerVersion
    Metadata:
      BuildMethod: python3.12
      BuildArchitecture: arm64
    Properties:
      ContentUri: layers/my_layer
      CompatibleArchitectures:
        - arm64
      LayerName: pipelines-opensearch

Adding some custom debug I see that a Layer class is called multiples times for MyLayer and at one point it sees the right configuration for CompatibleArchitectures but then it's gone

LayerVersion.__init__
            arn='MyLayer',
            codeuri='/source/layers/my_layer',
            compatible_runtimes=None,
            metadata={'BuildMethod': 'python3.12', 'BuildArchitecture': 'arm64', 'SamResourceId': 'MyLayer'},
            compatible_architectures=None,
            stack_path=''
LayerVersion.__init__ DONE

LayerVersion.__init__
            arn='MyLayer',
            codeuri='/source/layers/my_layer',
            compatible_runtimes=None,
            metadata={'BuildMethod': 'python3.12', 'BuildArchitecture': 'arm64', 'SamResourceId': 'MyLayer'},
            compatible_architectures=['arm64'],
            stack_path=''
LayerVersion.__init__ DONE

LayerVersion.__init__
            arn='MyLayer',
            codeuri='/source/layers/my_layer',
            compatible_runtimes=None,
            metadata={'BuildMethod': 'python3.12', 'BuildArchitecture': 'arm64', 'SamResourceId': 'MyLayer'},
            compatible_architectures=None,
            stack_path=''
LayerVersion.__init__ DONE

{'_stack_path': '', '_arn': 'MyLayer', '_codeuri': '/source/layers/my_layer', 'is_defined_within_template': True, '_metadata': {'BuildMethod': 'python3.12', 'BuildArchitecture': 'arm64', 'SamResourceId': 'MyLayer'}, '_build_method': 'python3.12', '_compatible_runtimes': [], '_custom_layer_id': 'MyLayer', '_build_architecture': 'arm64', '_compatible_architectures': None, '_skip_build': False, '_name': 'MyLayer', '_layer_id': 'MyLayer'}

Building layer 'MyLayer'
WARNING: Layer 'MyLayer' has BuildArchitecture arm64, which is not listed in CompatibleArchitectures
 Running PythonPipBuilder:ResolveDependencies
 Running PythonPipBuilder:CopySource
Building codeuri: functions/programs runtime: python3.12 metadata: {}
architecture: arm64 functions: ProgramsFunction
 Running PythonPipBuilder:ResolveDependencies
 Running PythonPipBuilder:CopySource

LayerVersion.__init__
            arn='MyLayer',
            codeuri='/source/layers/my_layer',
            compatible_runtimes=None,
            metadata={'BuildMethod': 'python3.12', 'BuildArchitecture': 'arm64', 'SamResourceId': 'MyLayer'},
            compatible_architectures=None,
            stack_path=''
LayerVersion.__init__ DONE

Observed result:

Check output at https://pastebin.com/raw/1PnPW63Z

Expected result:

Both of those cases should not happen.

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

1. OS: MacOs and Linux
2. sam --version: 1.120.0
3. AWS region: us-west-2

{
  "version": "1.120.0",
  "system": {
    "python": "3.12.4",
    "os": "macOS-14.5-arm64-arm-64bit"
  },
  "additional_dependencies": {
    "docker_engine": "24.0.9",
    "aws_cdk": "2.149.0 (build c8e5924)",
    "terraform": "1.6.2"
  },
  "available_beta_feature_env_vars": [
    "SAM_CLI_BETA_FEATURES",
    "SAM_CLI_BETA_BUILD_PERFORMANCE",
    "SAM_CLI_BETA_TERRAFORM_SUPPORT",
    "SAM_CLI_BETA_RUST_CARGO_LAMBDA"
  ]
}

[hnnasit]

Hi @cscetbon, thanks for opening the issue.

For issue 1, this seems like an expected behavior where a list defined in Globals would get merged with any specific value specified for a resource. You can read more about it here.

For issue 2, I tried to reproduce it but did not see that warning and got a successful build. Here's my setup:

template.yaml

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: >
  sam-app-7283

  Sample SAM Template for sam-app-7283

# Globals:
#   Function:
#     Architectures:
#       - arm64

Resources:
  ProgramsFunction:
    Type: AWS::Serverless::Function
    Properties:
      Architectures:
        - arm64
      CodeUri: hello_world/
      Handler: app.lambda_handler
      Layers:
        - !Ref MyLayer
      Runtime: python3.12

  MyLayer:
    Type: AWS::Serverless::LayerVersion
    Metadata:
      BuildMethod: python3.12
      BuildArchitecture: arm64
    Properties:
      ContentUri: layers
      CompatibleArchitectures:
        - arm64
      LayerName: pipelines-opensearch

project structure

.
├── README.md
├── __init__.py
├── events
│   └── event.json
├── hello_world
│   ├── __init__.py
│   ├── app.py
│   └── requirements.txt
├── layers
│   ├── __init__.py
│   ├── my_layer.py
│   └── requirements.txt
├── samconfig.toml
├── template.yaml

sam build output

hnnasit@bcd0744d58ec sam-app-7283 % sam build
Starting Build use cache                                                        
Manifest file is changed (new hash: 3298f13049d19cffaa37ca931dd4d421) or        
dependency folder (.aws-sam/deps/5ebffb22-c63b-4279-8f95-0a30a460a157) is       
missing for (MyLayer), downloading dependencies and copying/building source     
Building layer 'MyLayer'                                                        
 Running PythonPipBuilder:CleanUp                                               
 Running PythonPipBuilder:ResolveDependencies                                   
 Running PythonPipBuilder:CopySource                                            
 Running PythonPipBuilder:CopySource                                            
Manifest file is changed (new hash: 3298f13049d19cffaa37ca931dd4d421) or        
dependency folder (.aws-sam/deps/eefa2caf-aa08-471b-bfd2-9ef00181eb32) is       
missing for (ProgramsFunction), downloading dependencies and copying/building   
source                                                                          
Building codeuri:                                                               
/Users/hnnasit/Desktop/develop-workspace/sam-app-7283/hello_world runtime:      
python3.12 metadata: {} architecture: arm64 functions: ProgramsFunction         
 Running PythonPipBuilder:CleanUp                                               
 Running PythonPipBuilder:ResolveDependencies                                   
 Running PythonPipBuilder:CopySource                                            
 Running PythonPipBuilder:CopySource                                            

Build Succeeded

Built Artifacts  : .aws-sam/build
Built Template   : .aws-sam/build/template.yaml

Commands you can use next
=========================
[*] Validate SAM template: sam validate
[*] Invoke Function: sam local invoke
[*] Test Function in the Cloud: sam sync --stack-name {{stack-name}} --watch
[*] Deploy: sam deploy --guided

Is your layer not getting built?

[cscetbon]

For issue 1, this seems like an expected behavior where a list defined in Globals would get merged with any specific value specified for a resource. You can read more about it here.

I disagree on this, it says "If there are duplicates, the Resource section entry overrides the Globals section entry".

Is your layer not getting built?

It's getting built, it's only that the warning doesn't make sense.

[hnnasit]

I disagree on this, it says "If there are duplicates, the Resource section entry overrides the Globals section entry".

That's only true for Maps, Architectures here is a List and as per the docs,Lists are additive. An alternative to setting a global parameter could also be to use Parameters, which can be referenced in your lambda functions.

[cscetbon]

@hnnasit Oh I see, okay so on this one we're on the same page, thanks.