Using Buffer.from() with the contents of fs.readFileSync() makes a buffer with different data

let fb = fs.readFileSync(filePath);
let fileData = fb.toString("utf8");
b = Buffer.from(fileData, "utf8");
console.log(fb);
console.log(b);
console.log(b == fb);
console.log(b.toString("utf8") === fb.toString("utf8"));

I converted buffer to UTF8 string and again tried to get the buffer back using Buffer.from() but I can see that two buffer are different. However, if I again compare toString() result of these two buffers they are same.

Output of the above program

Answers:

Thank you for visiting the Q&A section on Magenaut. Please note that all the answers may not help you solve the issue immediately. So please treat them as advisements. If you found the post helpful (or not), leave a comment & I’ll get back to you as soon as possible.

Method 1

If fileData is not entirely legal UTF8, then per the buffer.toString() doc:

If encoding is ‘utf8’ and a byte sequence in the input is not valid UTF-8, then each invalid byte is replaced with the replacement character U+FFFD.

So, if you convert to a string and then back to a buffer and the encoding was not perfect utf8 to start with, then the process is not reversible because some illegal utf8 bytes will have been replaced.

Comparing two .toString('utf8') results will then each contain the same “fixed” version (with the replaced bytes).

The key here is don’t convert a Buffer to a utf8 string unless it’s really legal utf8 data.

All methods was sourced from stackoverflow.com or stackexchange.com, is licensed under cc by-sa 2.5, cc by-sa 3.0 and cc by-sa 4.0