How to get rid of quotes in strings when using yaml.dump

I am trying to use ruamel.yaml to generate cloudformation YAML templates from ordered dictionary. I need to prefix some of the strings with "!Sub" which is a cloudtemplate function reference in a way that it is not enclosed in quotes with the rest of the strings, e.g.:

Type: AWS::Glue::Job
Properties:
  Role2: !Sub 'arn:aws:iam::${AWS::AccountId}:role/${GlueServiceRole}'

Test code I started with looks like this:

from ruamel import yaml
from collections import OrderedDict

def ordered_dict_presenter(dumper, data):
    return dumper.represent_dict(data.items())

test = OrderedDict(
    {
        "Type": "AWS::Glue::Job",
        "Properties": {
            "Name": "test_job",
            "Role": "!Sub arn:aws:iam::${AWS::AccountId}:role/${GlueServiceRole}",
            "Role2": "Sub arn:aws:iam::${AWS::AccountId}:role/${GlueServiceRole}",
        },
    }
)
yaml.add_representer(OrderedDict, ordered_dict_presenter)

print(yaml.dump(test, default_style=None, default_flow_style=False))

It outputs this yaml:

Type: AWS::Glue::Job
Properties:
  Name: test_job
  Role: '!Sub arn:aws:iam::${AWS::AccountId}:role/${GlueServiceRole}'
  Role2: Sub arn:aws:iam::${AWS::AccountId}:role/${GlueServiceRole}

It seems like when a string starts with non-alphabetic character it is automatically quoted. I tried custom representers to get rid of quotes but without any luck so far. How can I output this?

1 answer

  • answered 2021-04-21 11:51 Anthon

    First of all you really shouldn't be using the old API anymore, there has been a new one, which allows much more control over what you are doing for several years now.

    Second, you cannot put something that looks like a tag inside a string and not get quotes, during loading that would load like a tag and you might want that, but that would make an infinite number of strings (all those starting with !) non-representable in YAML.

    If you don't know where to start, the thing to do is try and round-trip (load then dump) your required result:

    import sys
    import ruamel.yaml
    
    yaml_str = """\
    Type: AWS::Glue::Job
    Properties:
      Role2: !Sub arn:aws:iam::${AWS::AccountId}:role/${GlueServiceRole}
    """
    
    yaml = ruamel.yaml.YAML()
    yaml.preserve_quotes = True
    data = yaml.load(yaml_str)
    yaml.dump(data, sys.stdout)
    

    this gives:

    Type: AWS::Glue::Job
    Properties:
      Role2: !Sub arn:aws:iam::${AWS::AccountId}:role/${GlueServiceRole}
    

    This confirms that ruamel.yaml can create the output. Now you only need to do that from scratch. For which you can inspect data and especially data['Properties']['Role2'] (if things don't round-trip that often but not necessary, means you cannot generate what you want, but find out how to do it in that case can be harder).

    print(type(data['Properties']['Role2']))
    

    This prints:

    Type: AWS::Glue::Job
    Properties:
      Role2: !Sub arn:aws:iam::${AWS::AccountId}:role/${GlueServiceRole}
    <class 'ruamel.yaml.comments.TaggedScalar'>
    

    Now you only have to find out how TaggedScalar works ( source is in comments.py):

    import sys
    import ruamel.yaml
    
    data = {}
    data['Properties'] = props = {}
    props['Role2'] = ruamel.yaml.comments.TaggedScalar('arn:aws:iam::${AWS::AccountId}:role/${GlueServiceRole}', style=None, tag='!Sub')
    
    yaml = ruamel.yaml.YAML()
    yaml.preserve_quotes = True
    yaml.dump(data, sys.stdout)
    

    giving:

    Properties:
      Role2: !Sub arn:aws:iam::${AWS::AccountId}:role/${GlueServiceRole}
    

    Alternatively you can specify style="'" or style='"' if you do want to have quotes.