Filter out all occurrences of given properties from json object

I'm using PowerShell to extract data from an API call, update it and then pass it back to the API.

What I would like to know is whether or not there is a simple way to modify the JSON object, to filter out all the properties which are not desired at any location within the JSON structure?

I've tried the following, however the resultant JSON only has the lowest level properties removed (ie. "p2")

$example = ConvertFrom-Json '{"a":{"p1": "value1"},"p2": "value2", "b":"valueb"}'
$exclude = "p1", "p2"
$clean = $example | Select-Object -Property * -ExcludeProperty $exclude
ConvertTo-Json $clean -Compress

Result => {"a":{"p1":"value1"},"b":"valueb"}

I would like to have all $exlude entries removed, regardless of where they are located within the JSON. Is there a simple solution?

Update

Here is another (more complicated) JSON example:

{
  "a": {
    "p1": "value 1",
    "c": "value c",
    "d": {
      "e": "value e",
      "p2": "value 3"
    },
    "f": [
      {
      "g": "value ga",
      "p1": "value 4a"
      },
      {
      "g": "value gb",
      "p1": "value 4b"
      }
    ]
  },
  "p2": "value 2",
  "b": "value b"
}

The expected result (all p1 and p2 keys removed):

{
  "a": {
    "c": "value c",
    "d": {
      "e": "value e"
    },
    "f": [
      {
        "g": "value ga"
      },
      {
        "g": "value gb"
      }
    ]
  },
  "b": "value b"
}

1 answer

  • answered 2022-05-04 12:31 zett42

    Unfortunately there doesn't appear to be an easy way. My approach is to unroll the nested JSON properties, so we can easily apply filtering, then build a new object from the filtered properties.

    Steps one and three are wrapped in the following reusable helper functions, one for unroll (Expand-PSObjectRecursive) and one for rebuilding the object (Set-TreeValue):

    Function Expand-PSObjectRecursive {
        <#
        .SYNOPSIS
            Unroll a nested PSObject/PSCustomObject "property bag" such as created by ConvertFrom-JSON. 
        #>
        [CmdletBinding()]
        param (
            [Parameter(Mandatory, ValueFromPipeline)] [PSObject] $InputObject,
            [Parameter()] [string] $Path,
            [Parameter()] [string] $Separator = '.'
        )
        
        process {
            $InputObject.PSObject.Properties.ForEach{
    
                $propertyPath = if( $Path ) { "$Path$Separator$($_.Name)" } else { $_.Name }
            
                if( $_.Value -is [PSObject] ) {
                    # Recurse into child object
                    Expand-PSObjectRecursive $_.Value $propertyPath
                }
                else {
                    # Output the current property
                    [PSCustomObject]@{ 
                        Path = $propertyPath
                        Name = $_.Name
                        Value = $_.Value 
                    }
                }
            }
        }
    }
    
    function Set-TreeValue {
        <#
        .SYNOPSIS
            Set a property of a dictionary (tree) by path, creating nested child dictionaries as necessary. 
        #>
        [CmdletBinding()]
        param (
            [Parameter(Mandatory)] [System.Collections.IDictionary] $Tree,
            [Parameter(Mandatory, ValueFromPipelineByPropertyName)] [string] $Path,
            [Parameter(Mandatory, ValueFromPipelineByPropertyName)] $Value,
            [Parameter()] [string] $Separator = '.'
        )
    
        process {
            $subTree = $Tree
    
            do {
                # Split into root key and path remainder
                $key, $path = $path.Split( $Separator, 2 )
    
                if( $path ) {
                    # We have multiple path components, so we may have to create nested hash table.
                    if( -not $subTree.Contains( $key ) ) {
                        $subTree[ $key ] = [ordered] @{}
                    }           
                    # Enter sub tree. 
                    $subTree = $subTree[ $key ]
                }
                else {
                    # We have arrived at the leaf -> set its value
                    $subTree[ $key ] = $value
                }
            }
            while( $path )
        }
    }
    
    

    Now we can write:

    $example = ConvertFrom-Json '{"a": {"p1": "value 1", "c": "value c", "d": {"e": "value e", "p1": "value 3"}},"p2": "value 2", "b": "value b"}'
    $exclude = "p1", "p2"
    
    $clean = [ordered]@{}  # Create an ordered hashtable
    
    $example | Expand-PSObjectRecursive |    # Step 1: unroll (nested) properties
        Where-Object Name -notin $exclude |  # Step 2: apply filtering
        Set-TreeValue -Tree $clean           # Step 3: rebuild object
    
    $clean | ConvertTo-Json -Compress -Depth 9
    

    Output:

    {"a":{"c":"value c","d":{"e":"value e"}},"b":"value b"}
    

    Notes:

    • Child objects are removed if they don't contain any properties after filtering. So your first example would yield just '{"b":"valueb"}' after excluding p1 and p2 as a now is an empty object.
    • Set-TreeValue creates a nested hashtable, which ConvertTo-Json accepts. If you'd like to create an object similar to the input object, just convert the hashtable like this: $cleanObj = [PSCustomObject] $clean
    • Filtering can also be done on the whole path of a property. E. g. to remove the P1 property only within the a object, you could write Where-Object Path -ne a.p1.
    • Recursion in PowerShell functions is comparatively slow, because of the parameter-binding overhead. If performance is paramount, refer to this answer for an optimized Expand-PSObjectRecursive function (using inline C# and requiring PS Core 7+). There are other alternatives to avoid recursion altogether (e. g. using Collections.Queue), that work in older PS versions too. See this Gist if you want to go down that rabbit hole.

How many English words
do you know?
Test your English vocabulary size, and measure
how many words do you know
Online Test
Powered by Examplum