How can I capture the output of the Arduino CLI using Python?

I am experimenting with automating Arduino builds using Python's subprocess module. I cannot seem to capture the output from the compile/link process.

For example, if I call the Arduino CLI from my terminal using a sketch with a deliberate compiler error, I get this output:

james@james-pc:~/code/arduino-cli-test$ arduino --verify ~/sketchbook/test/test.ino
Picked up JAVA_TOOL_OPTIONS: 
Loading configuration...
Initialising packages...
Preparing boards...
Verifying...
sketch/test.cpp:7:25: fatal error: test.h: No such file or directory
 #include "test.h"
                  ^
compilation terminated.
exit status 1

But If I use subprocess to run the same command:

sketch_path="~/sketchbook/test/test.ino"
try:
    subprocess.run(["arduino", "--verify", sketch_path], check=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
except subprocess.CalledProcessError as e:
    print(e.output)
    print(e.stdout)
    print(e.stderr)

The except block runs, but I only get the output from Arduino itself, not the compiler output:

Out: 
Std: 
Err: Picked up JAVA_TOOL_OPTIONS: 
Loading configuration...
Initialising packages...
Preparing boards...
Verifying...

In this instance, both e.output and e.stdout were blank. I get different results there depending on the stdout and stderr parameters, but no combination of parameters I've tried has resulted in capturing the compiler output.

I get the same result if I eliminate check=True and examine the returned CompletedProcess object.

Why is this the case? Is there a way I can capture this output?

Footnote: Looking at this github issues and reading the documentation, maybe this isn't possible as the "arduino cli" is not very cli-like under the hood. Also, even redirect to file doesn't work, it just prints to the terminal:

james@james-pc:~/code/arduino-cli-test$ arduino --verify ~/sketchbook/test/test.ino > test.txt
Picked up JAVA_TOOL_OPTIONS: 
Loading configuration...
Initialising packages...
Preparing boards...
Verifying...
sketch/test.cpp:7:25: fatal error: test.h: No such file or directory
 #include "test.h"
                  ^
compilation terminated.
exit status 1
james@james-pc:~/code/arduino-cli-test$ cat test.txt 
james@james-pc:~/code/arduino-cli-test$