Read, then Write CSV with "Non-ISO extended-ASCII" text Encoding
My csv has strings like:
I'm trying to use the following below to set up a reader/writer
import csv # File that will be written to csv_output_file = open(file, 'w', encoding='utf-8') # File that will be read in csv_file = open(filename, encoding='utf-8', errors='ignore') # Define reader csv_reader = csv.reader(csv_file, delimiter=',', quotechar='"') # Define writer csv_writer = csv.writer(csv_output_file, delimiter=',', quotechar='"', quoting=csv.QUOTE_MINIMAL)
Then iterating over the information read in
# Iterate over the rows in the csv for idx, row in enumerate(csv_reader): csv_writer.writerow(row[0:30])
Problem is in my output file I can't get it to show up with that same string. According to my mac, the csv file type has the encoding "Non-ISO extended-ASCII"
I tried various encodings and some would just remove the special characters while others just wouldn't work.
It's weird because I can hard code that string above into a variable and use it without problems, so I assume it's something to do with how I'm reading in the file. If I breakpoint before it writes it shows up as the following in the debugger.
I can't convert the file before running it, so the python code has to handle any conversions itself.
The expected output I want would be for it to remain in the output file looking like
Edit: chardetect gave me "utf-8 with confidence 0.99"