NodeJS: How to read and modify buffer data, before parsing it from file?

In NodeJS, I am having a log file, where my logs are in the format:

{"time":"2021-09-23T11:36:18.076Z","type":"info","message":"some message","data":{"id":123}},
{"time":"2021-09-23T11:36:18.076Z","type":"info","message":"some message","data":{"id":123}},
{"time":"2021-09-23T11:36:18.076Z","type":"info","message":"some message","data":{"id":123}},

These are basically objects, separated with comma. What I need to do is to read the content of this file and to convert the logs into an array of objects (that I can manipulate later).

I am trying with something like:

    let fileLogs = "./data/myfile.log";
    fs.readFile(fileLogs, (err, fileLogsContent) => {
      if (err) {
        console.log("cannot read log file");
        return;
      }

      //I know I need to manipulate the fileLogsContent here, before doing JSON.parse

      let logsContent = { ...JSON.parse(fileLogsContent) };
      //do something here with the array of objects 'logsContent'
    });

Since the content in the log file is not in a format that can be parsed, the above JSON.parse fails. My idea is to bring the logfile in the following format:

[
{"time":"2021-09-23T11:36:18.076Z","type":"info","message":"some message","data":{"id":123}},
{"time":"2021-09-23T11:36:18.076Z","type":"info","message":"some message","data":{"id":123}},
{"time":"2021-09-23T11:36:18.076Z","type":"info","message":"some message","data":{"id":123}}
]

Which means that on the fly I would need to prepend [ as a first character and to replace the last , with ]. I am not aware how can I do that, since the fileLogsContent is actually a buffer. So how can I read the content and do the manipulations I mentioned, in order to be able to parse it later and to bring it into array of objects format?

1 answer

  • answered 2021-09-23 12:53 eol

    You can simply wrap each line in a string and then call JSON.parse on it after removing the trailing comma. Here's an example (note that it still needs error-handling ofc):

    const fs = require('fs');
    const readline = require('readline');
    const readInterface = readline.createInterface({
        input: fs.createReadStream('./input.txt'),
        output: undefined,
        console: false
    });
    
    (async () => {
        const resultArray = await new Promise((resolve, reject) => {
            const chunks = [];
            readInterface.on('line', (line) => {
                line = line.substr(0, line.lastIndexOf(','))
                chunks.push(JSON.parse(`${line}`));
            })
    
            readInterface.on('close', () => {
                resolve(chunks);
            })
        });
        console.log(resultArray); 
    })();
    

How many English words
do you know?
Test your English vocabulary size, and measure
how many words do you know
Online Test
Powered by Examplum