I am attempting to write a node app that reads in a set of files, splits them into lines, and puts the lines into an array. Pretty simple. It works on quite a few files except some SQL files that I am working with. For some reason I seem to be getting some kind of unicode output when I split the lines up. The app looks something like this:
Two questions here. I have a set of files which are usually UTF-8 with BOM. I’d like to convert them (ideally in place) to UTF-8 with no BOM. It seems like
codecs.StreamRecoder(stream, encode, decode, Reader, Writer, errors) would handle this. But I don’t really see any good examples on usage. Would this be the best way to handle this?
I have a UTF-16 CSV file which I have to read. Python csv module does not seem to support UTF-16.