In a json embedded YAML file - replace only json values using Python

I have a YAML file as follows:

api: v1
hostname: abc
metadata:
  name: test
  annotations: {
    "ip" : "1.1.1.1",
    "login" : "fad-login",
    "vip" : "1.1.1.1",
    "interface" : "port1",
    "port" : "443"
  }

I am trying to read this data from a file, only replace the values of ip and vip and write it back to the file.

What I tried is:

open ("test.yaml", w) as f:
    yaml.dump(object, f) #this does not help me since it converts the entire file to YAML

also json.dump() does not work too as it converts entire file to JSON. It needs to be the same format but the values need to be updated. How can I do so?

1 answer

  • answered 2022-05-07 07:00 Anthon

    What you have is not YAML with embedded JSON, it is YAML with some the value for annotations being in YAML flow style (which is a superset of JSON and thus closely resembles it).

    This would be YAML with embedded JSON:

    api: v1
    hostname: abc
    metadata:
      name: test
      annotations: |
        {
          "ip" : "1.1.1.1",
          "login" : "fad-login",
          "vip" : "1.1.1.1",
          "interface" : "port1",
          "port" : "443"
        }
    

    Here the value for annotations is a string that you can hand to a JSON parser.

    You can just load the file, modify it and dump. This will change the layout of the flow-style part, but that will not influence any following parsers:

    import sys
    import ruamel.yaml
    
    file_in = Path('input.yaml')
        
    yaml = ruamel.yaml.YAML()
    yaml.preserve_quotes = True
    yaml.width = 1024
    data = yaml.load(file_in)
    annotations = data['metadata']['annotations']
    annotations['ip'] = type(annotations['ip'])('4.3.2.1')
    annotations['vip'] = type(annotations['vip'])('1.2.3.4')
    yaml.dump(data, sys.stdout)
    

    which gives:

    api: v1
    hostname: abc
    metadata:
      name: test
      annotations: {"ip": "4.3.2.1", "login": "fad-login", "vip": "1.2.3.4", "interface": "port1", "port": "443"}
    

    The type(annotations['vip'])() establishes that the replacement string in the output has the same quotes as the original.

    ruamel.yaml currently doesn't preserve newlines in a flow style mapping/sequence. If this has to go back into some repository with minimal chances, you can do:

    import sys
    import ruamel.yaml
    
    file_in = Path('input.yaml')
    
    def rewrite_closing_curly_brace(s):
        res = []
        for line in s.splitlines():
            if line and line[-1] == '}':
                res.append(line[:-1])
                idx = 0
                while line[idx] == ' ':
                    idx += 1
                res.append(' ' * (idx - 2) + '}')
                continue
            res.append(line)
        return '\n'.join(res) + '\n'
        
    yaml = ruamel.yaml.YAML()
    yaml.preserve_quotes = True
    yaml.width = 15
    data = yaml.load(file_in)
    annotations = data['metadata']['annotations']
    annotations['ip'] = type(annotations['ip'])('4.3.2.1')
    annotations['vip'] = type(annotations['vip'])('1.2.3.4')
    yaml.dump(data, sys.stdout, transform=rewrite_closing_curly_brace)
    

    which gives:

    api: v1
    hostname: abc
    metadata:
      name: test
      annotations: {
        "ip": "4.3.2.1",
        "login": "fad-login",
        "vip": "1.2.3.4",
        "interface": "port1",
        "port": "443"
      }
    

    Here the 15 for width is of course highly dependent on your file and might influence other lines if they were longer. In that case you could leave that out, and make the wrapping that rewrite_closing_curly_brace() does split and indent the whole flow style part.

    Please note that your original, and the transformed output are, invalid YAML, that is accepted by ruamel.yaml for backward compatibility. According to the YAML specification the closing curly brace should be indented more than the start of annotation

How many English words
do you know?
Test your English vocabulary size, and measure
how many words do you know
Online Test
Powered by Examplum