Node CSV version 0.1 and future developments
The Node CSV parser has just reach version 0.1 which close the 0.0.x releases. Started almost 2 years ago, the project has received a tremendous amount of participation in the form of bug reports, pull request and emails. It is used by a large population of the NodeJs community and could now be considered as stable.
Well, this isn’t the best news. The really awesome news is that I have now some time to start the development of the v0.2 branch. This paves the way to some exciting developments which I am going to present in more details.
We say in french that we can’t prepare scramble eggs without breaking some eggs. So yes, the API will change, for the best. Since the beginning of the project, it felt wrong and the incomplete support of the stream API makes it non standard at best and, to be honest, hard to understand and use properly.
If your project rely on the 0.0.x or 0.1.x versions, don’t feel nervous. We will maintain support and any bug that we found will be ported to those versions.
This little schema illustrates the structure of the future code:
|------------| |------------|------------| |------------| | | | | | | | | csv.from | | CSV | | csv.to | | | | | | | | | Stream | | Writer | Reader | | Stream | | Reader | .pipe( | API | API | ).pipe( | Writer | ) | | | | | | | | | | | | | | |------------| |------------|------------| |------------|
A new CSV instance will be fully compliant with the stream API. It will be both an Stream Writer to send input data and a Stream Reader to access output data.
fs.createReadStream( './in' ) .pipe( csv() ) .pipe( fs.createWriteStream('./out') )
Alternatively, it will come with convenient functions accessible by the
to properties. Some of those functions were already present in the 0.1 release and are simply renamed. For exemple, the
csv.fromPath() function is now
csv.from.path(). New functions are planned such a the
csv.to.string and the
csv() .from.path( './in' ) .to.path( './out' )
Until now, all the source code present inside a single file. From now on, the code layout will be:
- ”./lib/from.js” Input convenient functions
- ”./lib/to.js” Output convenient functions
- ”./lib/csv.js” Core function implement the Stream API
- ”./lib/options.js” Default input and output options
- ”./lib/state.js” Default state object
This refactoring shall simplify the understanding of the source code and help new comers to prepare pull request which are particularly welcome.
Like I have done in the past in many projects like Mecano (now Nikita), the readme content will be reduced to a minimum and the documentation will be generated directly from the source code. A small script will be written specifically from that purpose. The idea is to document each function with comments written in a markdown syntax. A simple regexp parser read each files, extract the comment and write markdown file inside a “./doc” folder. The doc folder is finally copied into the Jekyll directory of this website before being uploaded.
I find it much easier to document a project on a per function basis than by maintaining coherent a doc forlder or a large readme file.
It is an excellent time to watch the development of this new version and come with your own feedback. I’ll be happy to listen to your suggestions.