Blazor WebAssembly PWA loses functionality and style in DevOps Pipeline

When first opening the application or after doing a hard reload the isolated CSS does not load. The same is if you open up the application on mobile. In addition, the promps to download the PWA doesn't show.

What I have is a Blazor WebAssembly PWA. The solution has two projects 'server' and 'client'.
The server project contains the start up, caching etc.
The client contains the frontend.

The solution is hosted in an Azure App Service (Deployment Slot).

If deploying manually to the slot everything is fine. (Download the publish profile from Azure AppService and publish server project)

Now to the problem:
I set up two pipelines, a build pipeline and a release pipeline.

The build pipeline:

trigger:
- master

pool:
  vmImage: 'windows-latest'

variables:
  solution: '**/*.sln'
  buildPlatform: 'Any CPU'
  buildConfiguration: 'Release'

steps:
- task: NuGetToolInstaller@1

- task: NuGetCommand@2
  inputs:
    restoreSolution: '$(solution)'

- task: VSBuild@1
  inputs:
    solution: '$(solution)'
    msbuildArgs: '/p:DeployOnBuild=true /p:WebPublishMethod=Package /p:PackageAsSingleFile=true /p:SkipInvalidConfigurations=true /p:PackageLocation="$(build.artifactStagingDirectory)"'
    platform: '$(buildPlatform)'
    configuration: '$(buildConfiguration)'

- task: VSTest@2
  inputs:
    platform: '$(buildPlatform)'
    configuration: '$(buildConfiguration)'

- task: PublishBuildArtifacts@1
  inputs:
    PathtoPublish: '$(Build.ArtifactStagingDirectory)'
    ArtifactName: 'drop'
    publishLocation: 'Container'

The release pipline (Has some CI/CD settings)

Due to building a solution with multiple project I get multiple Zip-files in my drop artifact. The server one is the one I pick up and deploy to the same AppService.

The pipelines work. The problem arises first when the application is opened. If I publish manually everything looks fine and I get the prompt to download the app. However, if I use the pipeline neither the correct CSS is loaded (at least not the isolated) or the prompt to download the PWA is shown.