Angular Material checkboxes with intermediate state based on JSON structure

I'm trying to implement tree menu with check boxes, that is based on object and the flow is supposed to go like:

unchecked -> checked (tick) -> intermediate (-) -> unchecked -> ...

But this is how it actually behaves:

unchecked -> checked (tick) -> intermediate (-) -> checked -> intermediate -> checked -> ...

The problem is, it never goes back to unchecked state. Problem demonstration:

Tags structure:

structuredTags = {
  group1: {
    tag1: { included: false, excluded: false },
    tag2: { included: false, excluded: false }
  group2: {
    tag3: { included: false, excluded: false },
    tag4: { included: false, excluded: false }

Items on the list have multiple tags groups, but I only care about tag names, not group names. I try to use [checked] and [intermediate] attributes like so:

    <li *ngFor="let group of structuredTags | keyvalue">
            <li *ngFor="let tag of group.value | keyvalue">
                <mat-checkbox (change)="toggleTag($event.checked, group.key, tag.key)"

I also tried using 2 sets and adding/deleting values to/from them and then in HTML use:

[checked]="selectedTags.has(tag.value)" [indeterminate]="excludedTags.has(tag.value)"

But it doesn't work as supposed to either. I think I'm missing something, any help will be much appreciated.