This tutorial explains how to modullarizing Node.js and Express.js applications. I am using the existing code from my previous post and explaining how to write a modular Node.js applications.

When it comes to writing the modular Node.js applications, there isn’t any one better way to write the modular applications. It comes through your experience and you may encounter different way to re-organize the code and every time it may look better approach for you. In this tutorial, I am going to explain which is better approach from my experience with Express.js and Node.js.

I have shown you how to build RESTful APIs using Express.js and Node.js here. In this post I will show you how we can modularize the code i.e keep related parts of the code in single Javascript file. For example, code dealing with DB can be placed in one javascript file, the code dealing with handling the requests can be placed in another Javascript file.

• RESTful Example using ExpressJS + Node.js + MongooseJS
• Mongoose – Node.js + MongoDB with Mongoose Tutorial
• Book Reference : Web Development with Node and Express

Writing Modular Node.js Applications

I am going to create the following modules from the example in this post:

1. Application Start point
2. Database Access Module
3. Controller module

Basic Introduction to Modularization in Node.js

Before I go a bit advanced into modularization of Node.js applications, I would like to take a very simple example to explain the basic concept behind modularizing code in Node.js. I will write a simple calculator app which supports: Addition, Subtraction, Multiplication. I will create a module for doing the calculation and another one which actually runs the application. So there would be 2 Javascript files namely: calculator.js and app.js.

Lets look at the calculator.js:

[code language=”javascript”]
//File name: calculator.js
//Use module.exports to export any function to be available via require()
module.exports.add = function(a,b){return a+b;}
module.exports.subtract = function(a,b){return a-b;}
module.exports.multiply = function(a,b){return a*b;}
[/code]

module is an Object in Node.js used to create Modules. We make the methods in calculator.js available by assigning them to the exports property of module object. We now have a module for performing calculations.

Note: Interesting read about difference between module.exports and exports.

Now lets use this module in our app.js which is the entry point for this simple sample to demonstrate basic concept behind modularity.

[code language=”javascript”]
var calculator = require("./calculator");
console.log("Sum of 4,5: "+ calculator.add(4,5));
console.log("Difference between 4,5: "+ calculator.subtract(4,5));
console.log("Product of 4,5: "+ calculator.multiply(4,5));
[/code]

Executing the above we would see something like below:

With this basic introduction to Modularity in Node.js let me show you how we can refactor the example in the post: RESTful Example using ExpressJS + Node.js + MongooseJS and modularize it.

Modularizing RESTful Example : Node.js Application

As stated at the beginning there would be a module for DB related code, a module for controller operations and then an application starting point. The directory structure of the sample looks like below:

Install the required packages i.e express, body-parser, mongoose using the command npm install express body-parser mongoose --save.

The code for the book_dao.js is as given below:

[code language=”javascript”]
//Filename: book_dao.js
//mongoose is used for interacting with MongoDB
var mongoose = require(‘mongoose’);

var dbHost = ‘mongodb://localhost:27017/test’;
mongoose.connect(dbHost);
//Create a schema for Book
var bookSchema = mongoose.Schema({
name: String,
//Also creating index on field isbn
isbn: {type: String, index: true},
author: String,
pages: Number
});

//Create a Model by using the schema defined above
//Optionally one can provide the name of collection where the instances
//of this model get stored. In this case it is "mongoose_demo". Skipping
//this value defaults the name of the collection to plural of model name i.e books.
var Book = mongoose.model(‘Book’, bookSchema);

//Connecting to Mongod instance.
mongoose.connection;

module.exports.findOne = function(isbn, callback){
Book.findOne({isbn: isbn}, function(err, result){
if ( err ) throw err;
callback(result);
});
}

module.exports.findAll = function(callback){
Book.find({}, function(err, result){
if ( err ) throw err;
callback(result);
});
}

module.exports.addNewBook = function(body, callback){
var book = new Book({
name:body.name,
isbn: body.isbn,
author: body.author,
pages: body.pages
});

//Saving the model instance to the DB
book.save(function(err, result){
if ( err ) throw err;
callback({
messaage:"Successfully added book",
book:result
});
});
}

module.exports.editBook = function(body, isbn, callback){
Book.findOne({isbn: isbn}, function(err, result){
if ( err ) throw err;

if(!result){
callback({
message:"Book with ISBN: " + isbn+" not found.",
});
}

result.name = body.name;
result.isbn = body.isbn;
result.author = body.author;
result.pages = body.pages;

result.save(function(err, result){
if ( err ) throw err;
callback({
message:"Successfully updated the book",
book: result
});
});

});
}

module.exports.deleteBook = function(isbn, callback){
Book.findOneAndRemove({isbn: isbn}, function(err, result){
callback({
message: "Successfully deleted the book",
book: result
});
});
}
[/code]

From the above code you can see that the book_dao.js depends only on the Mongoose API.

The code for book_controller.js is as given below:

[code language=”javascript”]
var bookDao = require("./book_dao");

module.exports.getBookDetails = function(params, callback){
console.log("Fetching details for book with ISBN: " + params.isbn);
bookDao.findOne(params.isbn, callback);
}

module.exports.getAllBooks = function(callback){
console.log("Fetching all books");
bookDao.findAll(callback);
}

module.exports.addNewBook = function(body, callback){
console.log("Adding new book");
bookDao.addNewBook(body, callback);
}

module.exports.editBook = function(body, isbn, callback){
console.log("Editing Book");
bookDao.editBook(body, isbn, callback);
}
module.exports.deleteBook = function(isbn, callback){
console.log("Deleting book");
bookDao.deleteBook(isbn, callback);
}
[/code]

The code for routes.js is given below:

[code language=”javascript”]
var bookController = require("./book_controller");

module.exports = function(app){
//Get the details of the book with the given isbn
app.get(‘/book/:isbn’, function(req, res){
bookController.getBookDetails(req.params, function(results){res.json(results);});
});

//Get all the books
app.get(‘/book’, function(req, res){
bookController.getAllBooks(function(results){res.json(results);});
});

app.post(‘/book’, function(req, res){
bookController.addNewBook(req.body, function(results){
res.json(results);
});
});

app.put(‘/book/:isbn’, function(req, res){
bookController.editBook(req.body, req.params.isbn, function(results){
res.json(results);
});
});

app.delete(‘/book/:isbn’, function(req, res){
bookController.deleteBook(req.params.isbn, function(results){
res.json(results);
});
});
}
[/code]

routes.js is responsible for mapping the URLs to the methods in the controller.

From above code its clear that book_dao.js depends only on Mongoose, book_controller.js depends only on the book_dao.js and routes.js depends only on book_controller.js

Running Modular Node.js Example Application

The below is the code for app.js which is the starting point for our application or in other words the place where the server is initiated and launched.

[code language=”javascript”]
//Filename: app.js
//Express is required for creating Node.js based web apps
var express = require(‘express’);

//body-parser is used to parse the Request body and populate the req.
var bodyParser = require(‘body-parser’);

var app = express();
app.set(‘port’, 3300);

//Configuring Express App to make use of BodyParser’s JSON parser to parse
//JSON request body
app.use(bodyParser.json());

//Including the routes module
var routes = require("./lib/routes");
routes(app);

//Starting up the server on the port: 3300
app.listen(app.get(‘port’), function(){
console.log(‘Server up: http://localhost:’ + app.get(‘port’));
});
[/code]

The above app.js depends on routes.js to setup the routes and controller binding.

Lets run the above sample as shown below:

[code language=”shell”]
G:\node\testapp\modular_sample_2>node app.js
Server up: http://localhost:3300
[/code]

The code for this sample can be found in the github as well.

• RESTful Example using ExpressJS + Node.js + MongooseJS
• Mongoose – Node.js + MongoDB with Mongoose Tutorial
• Book Reference : Web Development with Node and Express

I hope this tutorial helped you to understand how to modularize a Node.js application. This tutorial used RESTful example from our previous tutorials. Note that there is no one rule for modularizing the code, it can be improvized based on your experience. If you have any questions on writing the modular Node.js applications, please write it in the comments section.

In this tutorial I am going to explain you how to use the Node Package Manager (NPM) to package and deploy your own Node.js application. I assume that you have already installed the Node.js and NPM in your local system to run this example.

• Simple Steps to Install Node.js and NPM on Windows

In my previous post I showed you how to authenticate with Twitter and also make use of Twitter API to retrieve tweets. I have made use of several Node packages to achieve what I intended to. But what if someone wants to take the source code and run it in their machine? For example one of the readers have copied the source code and they now just want to run the application without knowing what all dependencies are there from the code and then installing them individually. Is this even possible in Node.js? Is there something like Maven where in we just mention the dependencies and the mvn automatically downloads the dependencies and can also launch the application.

The answer for the above questions is Yes, the solution for the above problmes is the Node Package Manager(NPM). Just like maven can host libraries which are reused in applications, even NPM can host libraries which are reused in your Node.js applications. NPM comes bundled with Node.js, so you can download Node.js from here and install it to also get NPM.

• RESTful Example using ExpressJS + Node.js + MongooseJS
• Mongoose – Node.js + MongoDB with Mongoose Tutorial
• Book Reference : Web Development with Node and Express

Now let us pick the sample app developed in this post, and create a deployable package which any user can download and run it in his environment.

The following Node packages are used in that sample app:

1. express
2. express-handlebars
3. request
4. querystring

Creating an Empty Node.js Code Base

I am creating a new directory by name: node_packaging and in that directory execute the following command:

[code language=”shell”]
npm init
[/code]

When you run the above command, after that it will ask you a series of questions (just like when you are initializing project using maven). The series of questions are asked so as to create and populate a new package.json file (think of it as your pom.xml in maven). If you want to know more about package.json and its attributes, visit here. The below is how it looks after initializing using NPM. Look at the below screen and input the answers which shown in the screenshot. These values are used by the NPM to create a package.json file.

After that you can see a file by name package.json created with the following content:

[code language=”javascript”]
{
"name": "node-packaging-demo",
"version": "1.0.0",
"description": "Demo to show node packaging",
"main": "social_signin.js",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1"
},
"author": "",
"license": "ISC"
}
[/code]

I can edit/update the package.json, for example add the author details as shown below:

[code language=”javascript”]
{
"name": "node-packaging-demo",
"version": "1.0.0",
"description": "Demo to show node packaging",
"main": "social_signin.js",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1"
},
"author": "Mohamed Sanaulla",
"license": "ISC"
}
[/code]

Adding Dependencies using NPM

All the dependencies of the application are declared in package.json. One either edit the file directly or make use of the npm tool to update the dependencies in package.json. The below code adds the required dependencies to our app:

[code language=”shell”]
npm install –save express express-handlebars request querystring
[/code]

The above command along with installing the mentioned packages, adds them as a dependency in pakcage.json. Let us now look at the contents of package.json:

[code language=”javascript” highlight=”11-16″]
{
"name": "node-packaging-demo",
"version": "1.0.0",
"description": "Demo to show node packaging",
"main": "social_signin.js",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1"
},
"author": "Mohamed Sanaulla",
"license": "ISC",
"dependencies": {
"express": "^4.12.4",
"express-handlebars": "^2.0.1",
"querystring": "^0.2.0",
"request": "^2.57.0"
}
}
[/code]

In the above the highlighted lines are the dependencies of our application and the number against it is the version of the package we are using. If no version is mentioned while executing npm install --save then the latest in NPM repository is used.

And when you list the files in your project folder you would see a folder by name node_modules. This is where all your modules get downloaded to.

Building the Node.js Application

The below is the directory structure of the application:

[code]
|–social_signin.js
|–views
|–home.handlebars
|–my.handlebars
|–layouts
|–main.handlebars
[/code]

This directory structure is relative to the directory we created first i.e node_packaging.

Following is the source code of our application

File: social_signin.js

[code language=”javascript”]
//FileName: social_signin.js
var express = require(‘express’);

//NPM Module to integrate Handlerbars UI template engine with Express
var exphbs = require(‘express-handlebars’);

//NPM Module to make HTTP Requests
var request = require("request");

//NPM Module To parse the Query String and to build a Query String
var qs = require("querystring");

var app = express();

//Declaring Express to use Handlerbars template engine with main.handlebars as
//the default layout
app.engine(‘handlebars’, exphbs({defaultLayout: ‘main’}));
app.set(‘view engine’, ‘handlebars’);

//URL To obtain Request Token from Twitter
var requestTokenUrl = "https://api.twitter.com/oauth/request_token";

//To be obtained from the app created on Twitter
var CONSUMER_KEY = "GET_IT_FROM_TWITTER";
var CONSUMER_SECRET = "GET_IT_FROM_TWITTER";

//Oauth Object to be used to obtain Request token from Twitter
var oauth = {
callback : "http://localhost:3000/signin-with-twitter",
consumer_key : CONSUMER_KEY,
consumer_secret : CONSUMER_SECRET
}
var oauthToken = "";
var oauthTokenSecret = "";
app.get(‘/’, function (req, res) {
//Step-1 Obtaining a request token
request.post({url : requestTokenUrl, oauth : oauth}, function (e, r, body){

//Parsing the Query String containing the oauth_token and oauth_secret.
var reqData = qs.parse(body);
oauthToken = reqData.oauth_token;
oauthTokenSecret = reqData.oauth_token_secret;

//Step-2 Redirecting the user by creating a link
//and allowing the user to click the link
var uri = ‘https://api.twitter.com/oauth/authenticate’
+ ‘?’ + qs.stringify({oauth_token: oauthToken})
res.render(‘home’, {url : uri});
});

});

//Callback to handle post authentication.
app.get("/signin-with-twitter", function(req, res){
var authReqData = req.query;
oauth.token = authReqData.oauth_token;
oauth.token_secret = oauthTokenSecret;
oauth.verifier = authReqData.oauth_verifier;

var accessTokenUrl = "https://api.twitter.com/oauth/access_token";
//Step-3 Converting the request token to an access token
request.post({url : accessTokenUrl , oauth : oauth}, function(e, r, body){
var authenticatedData = qs.parse(body);
console.log(authenticatedData);

//Make a request to get User’s 10 latest tweets
var apiUrl = "https://api.twitter.com/1.1/statuses/user_timeline.json" + "?"
+ qs.stringify({screen_name: authenticatedData.screen_name, count: 10});

var authenticationData = {
consumer_key : CONSUMER_KEY,
consumer_secret : CONSUMER_SECRET,
token: authenticatedData.oauth_token,
token_secret : authenticatedData.oauth_token_secret
};

request.get({url : apiUrl, oauth: authenticationData, json:true}, function(e, r, body){

var tweets = [];
for(i in body){
var tweetObj = body[i];
tweets.push({text: tweetObj.text});
}

var viewData = {
username: authenticatedData.screen_name,
tweets: tweets
};

res.render("my", viewData);

});

});
});

app.listen(3000, function(){
console.log(‘Server up: http://localhost:3000’);
});
[/code]

File Name: main.handlebars

[code language=”html”]
<!– FileName: main.handlebars–>
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<title>Example App</title>
</head>
<body>
{{{body}}}
</body>
</html>
[/code]

File Name: home.handlebars

[code language=”html”]
<!– FileName: home.handlebars–>
<h1>SignIn With Twitter Sample</h1>

<a href="{{url}}">Signin with Twitter</a>
[/code]

File Name: my.handlebars

[code language=”html”]
<!– FileName: my.handlebars–>
<h1>Welcome {{username}} </h1>
<h3>Your tweets</h3>
<ul>
{{#tweets}}
<li>{{text}}</li>
{{/tweets}}
</ul>
[/code]

Running the Node.js Sample Application

[code]
G:\node\node_packaging>node social_signin.js
Server up: http://localhost:3000
[/code]

Navigating to http://localhost:300 gives use a page with “Sign with Twitter” link. Clicking on it shows us the latest 10 tweets.

Pushing the code to Github

Now I am going to push the code to Github and then show you how packaging using NPM and package.json helps in sharing the application among other developers. Before carrying out the below task, install Git on your machine.

Carry out the following steps to push the code to github:

1. Create a new repository on github.com
2. Run git init in your project folder to initialize a new git repository locally.
3. Run git status in your project folder, you will see the following output:

   [code language=”shell”]
   $ git status
   On branch master

   Initial commit

   Untracked files:
   (use "git add <file>…" to include in what will be committed)

   node_modules/
   package.json
   social_signin.js
   views/

   nothing added to commit but untracked files present (use "git add" to track)
   [/code]

   We need to remove node_modules folder from getting committed to Github. So we create a new .gitignore file in the project root folder i.e in node_packaging folder. The content of .gitignore file is:

   [code]
   $ cat .gitignore
   node_modules
   [/code]

   Run git status again, this time you will see that the node_modules folder is not listed in the files to be committed:

   [code]
   $ git status
   On branch master

   Initial commit

   Untracked files:
   (use "git add <file>…" to include in what will be committed)

   .gitignore
   package.json
   social_signin.js
   views/

   nothing added to commit but untracked files present (use "git add" to track)
   [/code]

4. Add the Github repository you created in Step-1 as a remote repository to the local repository you created in Step-2. This can be done using the following command: git remote add origin GITHUB_REPO_URL. You can get the GITHUB_REPO_URL from GitHub Repository page you created in Step-1
5. Now commit and push your changes to the Github repository. It can be done as follows:

   [code]
   $ git add -A
   $ git commit
   $ git push origin master
   [/code]

Now the code should be available in your Github repository. Mine is at: https://github.com/sanaulla123/nodejs-package-demo

Advantage of Node.js Packaging

So we have setup everything for any developer to just clone the Github repository and start running the application locally. Let us also try to clone the repository in another location and see if this packaging really works!!

[code language=”shell”]
Mohamed@SANA-LAPTOP /g/node/node_packaging (master)
$ cd ..

Mohamed@SANA-LAPTOP /g/node
$ mkdir node_packaging_clone

Mohamed@SANA-LAPTOP /g/node
$ cd node_packaging_clone/

#Cloning the Github repository
Mohamed@SANA-LAPTOP /g/node
$ git clone https://github.com/sanaulla123/nodejs-package-demo.git

Mohamed@SANA-LAPTOP /g/node/node_packaging_clone
$ ls
nodejs-package-demo

Mohamed@SANA-LAPTOP /g/node/node_packaging_clone
$ cd nodejs-package-demo/

#There is no node_modules folder!!
Mohamed@SANA-LAPTOP /g/node/node_packaging_clone/nodejs-package-demo (master)
$ ls
package.json social_signin.js views

#This command picks up the dependencies from package.json and installs them to the local project folder.
Mohamed@SANA-LAPTOP /g/node/node_packaging_clone/nodejs-package-demo (master)
$ npm install

#The node_modules folder got created after the above command.
Mohamed@SANA-LAPTOP /g/node/node_packaging_clone/nodejs-package-demo (master)
$ ls
node_modules package.json social_signin.js views
[/code]

Now you should be able to run the application as before. This was a small introduction to packaging your node.js applications. Hope you found it easy to understand and beneficial. If you have any questions on Node.js Application Packaging, please write it in the comments section. Enjoy reading more tutorials about Node.js in our blog.