Automating Markdown and FTP with Gulp

One of the issues with rolling your own blog system is friction. Most pre-made bogging system have devoted lots of effort to making new posts as easy and simple as possible. My blog is still pretty rough around the edges and making new posts is kind of a hassle. Until this weekend when I decided to see how I could automate the process and reduce the friction in creating new posts.

Quick side note: if you want a good explanation and guide on Gulp, I recommend this one.

Gulp and Pandoc

I’ve been using Pandoc to convert my posts from Markdown to HTML. Markdown is quick and easy to write in and VS Code has great support. I tried to install some NPM modules for the conversion, but NPM and Windows’ max-path length causes issues so I went with Pandoc.

Before this weekend I’d convert a post like so:

> pandoc posts\src\2016-06-19-gulp-pandoc-ftp.md -o posts\2016-06-19-gulp-pandoc-ftp.html

While that handled the conversion very well, it required me to run it after each save, or whenever I wanted to see it in the browser.

Enter Gulp and gulp-pandoc. I have a task that takes in all of my post files, pipes them through Pandoc, then writes them out to the correct folder.

gulp.task('posts', function () {
    gulp.src('www/posts/src/*.md')
        .pipe(pandoc({
            from: 'markdown',
            to: 'html5',
            ext: '.html',
            args: ['--smart']
        }))
        .pipe(gulp.dest('www/posts'))
        .pipe(browserSync.reload({
            stream: true
        }));
});

I also use BrowserSync to create a simple web server and have a task with a watch that automatically converts the posts when I update the Markdown files.

Gulp and FTP

The next area of friction is getting the files to the server. To put it simply, all FTP clients kind of suck. If you know of a good Windows one (preferably UWP), let me know on Twitter. Automating it would be the best option. Enter vinyl-ftp:

gulp.task('deploy', function () {
    var conn = ftp.create({
        host: '***.**********.***',
        user: '********',
        password: '********',
        parallel: 10,
        log: gutil.log
    });

    gulp.src('www/**/*', { base: '.', buffer: false })
        .pipe(rename(function (path) {
            path.dirname = path.dirname.replace('www', '');

            return path;
        }))
        .pipe(conn.newer('/public_html'))
        .pipe(debug({ title: 'Files: ' }))
        .pipe(conn.dest('/public_html'));
});

The best part of this is that vinyl-ftp can determine which files have changed and only upload those. I would love to apply that kind of logic to the Pandoc task (it just converts all the posts every time).

Conclusion

I’ve now automated the most annoying parts of my blogging process. There’s clearly more to do, but it is a great start and a fun little exploration into new (for me) Gulp plugins.

Update: I don’t use either of these anymore. I rewrote this whole blog using Hexo. Read about that here.