JavaBeat

  • Home
  • Java
    • Java 7
    • Java 8
    • Java EE
    • Servlets
  • Spring Framework
    • Spring Tutorials
    • Spring 4 Tutorials
    • Spring Boot
  • JSF Tutorials
  • Most Popular
    • Binary Search Tree Traversal
    • Spring Batch Tutorial
    • AngularJS + Spring MVC
    • Spring Data JPA Tutorial
    • Packaging and Deploying Node.js
  • About Us
    • Join Us (JBC)
  • Privacy
  • Contact Us

MongoDB : GridFS Tutorial

September 10, 2015 by Mohamed Sanaulla Leave a Comment

This GridFS tutorial explains how to use GridFS for stroring the larger files in MongoDB and when to use GridFS in MongoDB. This GridFS tutorial stores image file of 16.5 MB size to illustrate an example.

MongoDB limits the size of the document in a collection to 16 MB. Realistically this should be good enough for data which is a group of key-value pairs. But when you want to store files along with this key-value pairs, we often find the upper limit of 16 MB to be a limitation. To overcome this, MongoDB provides an API called GridFS where we can store files greater than 16 MB along with any metadata related to the file.

In this post we will look at how we can store and retrieve files from and to mongodb using GridFS and MongoDB java driver. Let us first look at the basics of GridFS.

GridFS Tutorial

gridfs

Basics of GridFS

GridFS stores the files in the form of a series of chunks where by default each chunk size can be of maximum 255k. There are two collections namely files and chunks. The files collection stores the metadata of the file and chunks collection stores the chunks of the files with each chunk having file_id and n where file_id is the _id of the parent chunk and n is the chunk number.

The collections files and chunks are stored under a namespace. By default the namespace is fs. We can override the namespace and provide our own.

Saving files to Mongodb using GridFS

Let us first look at saving file to Mongodb using GridFS. For this I am going to consider an image with size 16.5MB.

  • Also Read : Node.js + MongoDB – Performing CRUD Operations

Note: You can download any image file for your use which is large in size.
The below code saves the image at a given location to Mongodb using GridFS:

[code lang=”java”]
import java.io.File;
import java.io.IOException;
import com.mongodb.DB;
import com.mongodb.DBCollection;
import com.mongodb.MongoClient;
import com.mongodb.gridfs.GridFS;
import com.mongodb.gridfs.GridFSInputFile;

public class GridFsSaveDemo {

public static void main(String[] args) throws IOException {
MongoClient mongo = new MongoClient("localhost", 27017);
DB db = mongo.getDB("filesdb");

//Location of file to be saved
String imageLocation = "C:/Users/Mohamed/Pictures/image2.jpg";

//Create instance of GridFS implementation
GridFS gridFs = new GridFS(db);

//Create a file entry for the image file
GridFSInputFile gridFsInputFile = gridFs.createFile(new File(imageLocation));

//Set a name on GridFS entry
gridFsInputFile.setFilename("image1");

//Save the file to MongoDB
gridFsInputFile.save();
}
}
[/code]

Let us open mongo shell to verify if the above file got saved:

[code lang=”shell”]
> use filesdb
switched to db filesdb
> show collections
fs.chunks
fs.files
system.indexes
> db.fs.chunks.find().count()
67
> db.fs.files.find().pretty()
{
"_id" : ObjectId("55e3279f355311259428f3a9"),
"filename" : "image1",
"aliases" : null,
"chunkSize" : NumberLong(261120),
"uploadDate" : ISODate("2015-08-30T15:56:15.017Z"),
"length" : NumberLong(17315594),
"contentType" : null,
"md5" : "a6bc0171f7beff8036715dd1d022c1a0"
}
[/code]

You can notice above that the file was divided into 67 chunks stored in fs.chunks collection and file metadata i.e the fix size, file name, upload date stored in fs.files collection.

Reading files from MongoDB using GridFS

The code for reading the image saved above is given below:

[code lang=”java”]
import java.io.IOException;
import com.mongodb.DB;
import com.mongodb.MongoClient;
import com.mongodb.gridfs.GridFS;
import com.mongodb.gridfs.GridFSDBFile;

public class GridFsReadDemo {
public static void main(String[] args) throws IOException {
MongoClient mongo = new MongoClient("localhost", 27017);
DB db = mongo.getDB("filesdb");

//Create instance of GridFS implementation
GridFS gridFs = new GridFS(db);

//Find the image with the name image1 using GridFS API
GridFSDBFile outputImageFile = gridFs.findOne("image1");

//Get the number of chunks
System.out.println("Total Chunks: " + outputImageFile.numChunks());

//Location of the image read from MongoDB to be written
String imageLocation = "C:/Users/Mohamed/Pictures/mongoImage.jpg";
outputImageFile.writeTo(imageLocation);
mongo.close();

}
}
[/code]

The above code queries MongoDB to find the image chunks by using the image name. And then writes those image chunks to file system.

  • Also Read : Spring Boot : RESTful API using Spring Boot and MongoDB

GridFS makes it very easy to store and retrieve files from MongoDB. We can use it to store not only files greater then 16MB but also store files lesser than 16MB.

I hope this GridFS tutorial helped you to understand how to store the files larger than 16 MB and storing an image file in the MongoDB database.

Filed Under: MongoDB Tagged With: GridFS Tutorials

Spring Boot : RESTful API using Spring Boot and MongoDB

June 25, 2015 by Mohamed Sanaulla Leave a Comment

This tutorial explains how to implement RESTful web service using SpringBoot and MongoDB. This tutorials uses already implemented REST services using Node.js and ExpressJS frameworks. Here the same examples tested with SpringBoot.

The following are the frameworks used in this tutorial for running the given example. If you are very much interested in learning the REST Web Services concepts, please visit our pal Eugen’s world class video course on BUILD YOUR REST API WITH SPRING.

  • Spring Boot
  • MongoDB
  • ExpressJS
  • Node.js
  • RESTful Concepts

In this post I will implement the REST API designed in one of my previous posts here.

Create RESTful API using Spring Boot and MongoDB
Create RESTful API using Spring Boot and MongoDB

Creating a simple Maven project in Eclipse

The following steps helps you to create a simple Maven project in Eclipse. We have several good tutorials on Maven for the beginners. If you are looking for interesting reads on Maven, please read Apache Maven for Beginners, Create Web Application Project with Maven and How to Build Spring MVC Application with Maven.

Project Type selection

SpringBoot MongoDB RESTAPI
SpringBoot MongoDB RESTAPI

Workspace and archetype selection

SpringBoot MongoDB RESTAPI MAVEN
SpringBoot MongoDB RESTAPI MAVEN

Maven project properties
SpringBoot MongoDB RESTAPI MAVEN 1

The final project structure
SpringBoot MongoDB RESTAPI MAVEN Project

Now that the basic maven based Spring boot is ready, let me give you an overview of Springboot and the idea behind it.

REST API using Spring Boot and MongoDB

What is Spring Boot?

Spring Boot is approach to develop Spring based application with very less configuration. It leverages existing Spring projects as well as Third party projects to develop production ready applications. It provides a set of Starter Pom’s or gradle build files which one can use to add required dependencies and also facilitate auto configuration.

Depending on the libraries on its classpath, Spring Boot automatically configures required classes. For example to interact with DB, if there is Spring Data libraries on class path then it automatically sets up connection to DB along with the Data Source class.

More about Spring Boot can be found here.

Creating RESTful Web Service

I will not show you how to create a RESTful Web Service using Spring Boot. The API which I am going to implement in this post has already been defined in one of my previous posts here.

spring-boot-starter-parent

Lets update the pom.xml to add refer to the parent pom which is: spring-boot-starter-parent. This is the starter pom provided by Spring Boot project. This start pom:

  • sets up the maven properties
  • basic dependency to spring-core
  • setting up location for resource files
  • setting up maven plugins including the plugin provided by spring boot to run its application

[code language=”xml” highlight=”8-15″]
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>net.javabeat</groupId>
<artifactId>springboot-rest-demo</artifactId>
<version>0.1</version>
<name>Spring Boot REST API Demo</name>
<properties>
<java.version>1.8</java.version>
</properties>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>1.2.3.RELEASE</version>
</parent>
</project>
[/code]

spring-boot-starter-web

In order to build RESTful Service we would need make use of: spring web, JSON processing libraries, embedded tomcat or tomcat libraries, may be some library for validators like Hibernate validator. We might not remember all the dependencies required or might have to refer to some place to know which dependencies are required.

To solve such issues Spring Boot provides a pom already configured with the dependencies required to build a web application and that pom is called: spring-boot-starter-web. Updating our pom.xml to add it as a dependency we get:

[code language=”xml” highlight=”16-21″]
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>net.javabeat</groupId>
<artifactId>springboot-rest-demo</artifactId>
<version>0.1</version>
<name>Spring Boot REST API Demo</name>
<properties>
<java.version>1.8</java.version>
</properties>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>1.2.3.RELEASE</version>
</parent>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
</dependencies>
</project>
[/code]

Similarly we have a pom: spring-boot-starter-data-mongodb which includes dependencies to mongo java driver, spring transaction, spring data for mongodb. So our final pom.xml looks like:

[code language=”xml” highlight=”21-24″]
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>net.javabeat</groupId>
<artifactId>springboot-rest-demo</artifactId>
<version>0.1</version>
<name>Spring Boot REST API Demo</name>
<properties>
<java.version>1.8</java.version>
</properties>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>1.2.3.RELEASE</version>
</parent>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-mongodb</artifactId>
</dependency>
</dependencies>
</project>
[/code]

Spring Boot Application

We need to create a main class which initializes the Spring Boot application and runs it. We also annotate the class with @SpringBootApplication. This annotation is tells the Spring application to enable autoconfiguration and component scan and also tells the Spring application that this particular class is also a configuration:

[code lang=”java”]
package app;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

@SpringBootApplication
public class Application {

public static void main(String[] args) throws Exception {
SpringApplication.run(new Object[] { Application.class }, args);
}
}
[/code]

Implementing the RESTful APIs

Let me create a model class to hold Book details retrieved from DB.

[code language=”java”]
package app.model;

import org.springframework.data.annotation.Id;

public class Book {

@Id
private String id;
private String name;
private String isbn;
private String author;
private int pages;

public Book(){}

public Book(String name, String isbn, String author, int pages){
this.name = name;
this.isbn = isbn;
this.author = author;
this.pages = pages;
}

public String getId() {
return id;
}
public void setId(String id) {
this.id = id;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public String getIsbn() {
return isbn;
}
public void setIsbn(String isbn) {
this.isbn = isbn;
}
public String getAuthor() {
return author;
}
public void setAuthor(String author) {
this.author = author;
}
public int getPages() {
return pages;
}
public void setPages(int pages) {
this.pages = pages;
}
}

[/code]

Lets add Repository class to interact with the DB

[code language=”java”]
package app.repository;

import app.model.Book;

import org.springframework.data.mongodb.repository.MongoRepository;

public interface BookRepository extends MongoRepository<Book, String>{
}
[/code]

The MongoRepository provides basic CRUD operation methods and also an API to find all documents in the collection.

Implementing Create and Get Details API

Lets implement our controller. First is the API to create new book and also to get the details of a given book:

[code language=”java”]
package app.controller;
import java.util.LinkedHashMap;
import java.util.Map;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.RestController;

import app.model.Book;
import app.repository.BookRepository;

@RestController
@RequestMapping("/book")
public class BookController {

@Autowired
private BookRepository bookRepository;

@RequestMapping(method = RequestMethod.POST)
public Map<String, Object> createBook(@RequestBody Map<String, Object> bookMap){
Book book = new Book(bookMap.get("name").toString(),
bookMap.get("isbn").toString(),
bookMap.get("author").toString(),
Integer.parseInt(bookMap.get("pages").toString()));

bookRepository.save(book);
Map<String, Object> response = new LinkedHashMap<String, Object>();
response.put("message", "Book created successfully");
response.put("book", book);
return response;
}

@RequestMapping(method = RequestMethod.GET, value="/{bookId}")
public Book getBookDetails(@PathVariable("bookId") String bookId){
return bookRepository.findOne(bookId);
}
}
[/code]

Lets run the application by either using Spring Boot maven plugin i.e by running mvn spring-boot:run or by running the main class Application.java from Eclipse or your IDE. I will use Postman REST Client to invoke the create API:
1

The below is the screenshot from Postman to invoke the Get Book Details API:
2

Implementing Update API

Now lets implement the Update API (just showing the method implemented here, it should be part of the BookController class defined above):

[code language=”java”]
@RequestMapping(method = RequestMethod.PUT, value="/{bookId}")
public Map<String, Object> editBook(@PathVariable("bookId") String bookId,
@RequestBody Map<String, Object> bookMap){
Book book = new Book(bookMap.get("name").toString(),
bookMap.get("isbn").toString(),
bookMap.get("author").toString(),
Integer.parseInt(bookMap.get("pages").toString()));
book.setId(bookId);

Map<String, Object> response = new LinkedHashMap<String, Object>();
response.put("message", "Book Updated successfully");
response.put("book", bookRepository.save(book));
return response;
}
[/code]

Screenshot from Postman REST client after executing the Update:
3

Implementing the Delete API

[code language=”java”]
@RequestMapping(method = RequestMethod.DELETE, value="/{bookId}")
public Map<String, String> deleteBook(@PathVariable("bookId") String bookId){
bookRepository.delete(bookId);
Map<String, String> response = new HashMap<String, String>();
response.put("message", "Book deleted successfully");

return response;
}
[/code]

Screenshot from Postman REST client after executing the Delete:
4

Implementing the Get All Books API

[code language=”java”]
@RequestMapping(method = RequestMethod.GET)
public Map<String, Object> getAllBooks(){
List<Book> books = bookRepository.findAll();
Map<String, Object> response = new LinkedHashMap<String, Object>();
response.put("totalBooks", books.size());
response.put("books", books);
return response;
}
[/code]

Screenshot from Postman REST client after executing the Get all books:

5

With this I have shown you how to build a REST API for CRUD operations using Spring Boot and MongoDB. One can see that with minimum configuration I have been able to write a full fledged implementation of REST API. I hope this tutorial have helped to understand how to write Spring Boot application with MongoDB.

  • Professional Node.js
  • Node.js in Action
  • Node.js Receipes

MongoDB is the popular NoSQL database and Spring Boot is the latest framework from Spring community to run the Spring applications with minimal configurations. Hence, Spring Boot and MongoDB makes good combination to work together for building the RESTful APIs.

If you have any questions on how to write Spring Boot application with MongoDB, please write it in the comments section. Also you can access my Github below for the complete code for Spring Boot and MongoDB integration example. Have a happy reading!!!

The project can be found on Github here.

Filed Under: MongoDB, Spring Framework Tagged With: SpringBoot Tutorials

RESTful Example using ExpressJS + Node.js + MongooseJS

May 18, 2015 by Mohamed Sanaulla Leave a Comment

RESTful Web Services Example : ExpressJS + Node.js + MongooseJS

In my previous articles I have explaines about CRUD operations Node.js + MongoDB – Performing CRUD Operations and Mongoose – Node.js + MongoDB with Mongoose Tutorial. In this post I will show you how to create CRUD operations using RESTful Web Services and ExpressJS with MongooseJS. I would be using the following 3 Node packages for this sample namely:

  1. ExpressJS
  2. body-parser
  3. MongooseJS

ExpressJs

From the Node Package manager website:

Fast, unopinionated, minimalist web framework

Express is used to create Web applications using Node.js. It provides support for routes, setting up view templates and others. This is most popular Node.js web framework as of now. The advantage of using ExpressJS is to reduce the time and lines of code written unless using the plain Node.js modules.

Body-parser

From the Node Package manager wwebsite:

Node.js body parsing middleware.

This middleware is used for intercepting the requests, parsing the request body and then populating it in the node.js’s req object.

MongooseJS

From the Node Package Manager website:

MongooseJS is a MongoDB object modeling tool designed to work in an asynchronous environment.

This is used to define Model objects and then use it to interact with MongoDB, a kind of ORM in the OO Programming world. MongooseJS is a Object Data Mapping (ODM) for JavaScript frameworks. This helps you to map the JavaScript objects to the MongoDB database. This tool is developed and maintained by the MongoDB community itself.

enm

Defining the RESTful API

In this tutorial I am going to implement a simple RESTful Web Services (Read : Spring REST Web Services) example using ExpressJS ,Node.js and MongooseJS. The right below section just defines the list of operations that are going to implemented as part of the excersise. The list of RESTful APis are :

  • Fetch list of all books in the system : GET /book/
  • Fetch details of a book in the system : GET /book/:isbn
  • Add a new book to the system : POST /book/
  • Edit details of existing book in the system : PUT /book/:isbn
  • Delete given book from the system : DELETE /book/:isbn

1. Fetch list of all books in the system : GET /book/
Parameters: No parameters
Response: The response is going to be an array of JSON objects (Also Read : What is JSON?) with the structure given below.

[code language=”javascript”]
[
{
"_id":STRING,
"name":STRING,
"isbn":STRING,
"author":STRING,
"pages":INTEGER,
"_v":STRING
}
]
[/code]

2. Fetch details of a book in the system : GET /book/:isbn
Parameters: ISBN of the book
Response: The response is going to be the JSON object with the structure given below.

[code language=”javascript”]
{
"_id":STRING,
"name":STRING,
"isbn":STRING,
"author":STRING,
"pages":INTEGER,
"_v":STRING
}
[/code]

3. Add a new book to the system : POST /book/
Parameters: Data of following format has to be sent in the request body

[code language=”javascript”]
{
"name":STRING,
"isbn":STRING,
"author":STRING,
"pages":INTEGER
}
[/code]

Response: The response is going to be the JSON object with the structure given below.

[code language=”javascript”]
{
"messaage": "Successfully added book",
"book": {
"__v": STRING,
"name": STRING,
"isbn": STRING,
"author": STRING,
"pages": INTEGER,
"_id": STRING
}
}
[/code]

4. Edit details of existing book in the system : PUT /book/:isbn
Parameters: ISBN of the book and the updated book information in the below format in request body.

[code language=”javascript”]
{
"name":STRING,
"isbn":STRING,
"author":STRING,
"pages":INTEGER
}
[/code]

Response: The response is going to be the JSON object with the structure given below.

[code language=”javascript”]
{
"messaage": "Successfully updated the book",
"book": {
"__v": STRING,
"name": STRING,
"isbn": STRING,
"author": STRING,
"pages": INTEGER,
"_id": STRING
}
}
[/code]

5. Delete given book from the system : DELETE /book/:isbn
Parameter: ISBN of the book to be deleted
Response: The response is going to be the JSON object with the structure given below.

[code language=”javascript”]
{
"messaage": "Successfully deleted the book",
"book": {
"__v": STRING,
"name": STRING,
"isbn": STRING,
"author": STRING,
"pages": INTEGER,
"_id": STRING
}
}
[/code]

Implementing the RESTful API : ExpressJS + Body-parser + Node.js

As mentioned earlier I would be using express.js, node.js and body-parser.js node packages with MongooseJS. The below code sets up the packages and starts the web server on port 3300.

[code language=”javascript”]
//Express is required for creating Node.js based web apps
var express = require(‘express’);

//body-parser is used to parse the Request body and populate the req.
var bodyParser = require(‘body-parser’);

//mongoose is used for interacting with MongoDB
var mongoose = require(‘mongoose’);

var app = express();
app.set(‘port’, 3300);

//Configuring Express App to make use of BodyParser’s JSON parser to parse
//JSON request body
app.use(bodyParser.json());

var dbHost = ‘mongodb://localhost:27017/test’;
mongoose.connect(dbHost);
//Create a schema for Book
var bookSchema = mongoose.Schema({
name: String,
//Also creating index on field isbn
isbn: {type: String, index: true},
author: String,
pages: Number
});

//Create a Model by using the schema defined above
//Optionally one can provide the name of collection where the instances
//of this model get stored. In this case it is "mongoose_demo". Skipping
//this value defaults the name of the collection to plural of model name i.e books.
var Book = mongoose.model(‘Book’, bookSchema);

//Connecting to Mongod instance.
mongoose.connection;

//Starting up the server on the port: 3300
app.listen(app.get(‘port’), function(){
console.log(‘Server up: http://localhost:’ + app.get(‘port’));
});
[/code]

You can save the above code in file: express_demo.js and then run it using node express_demo.js , you should be able to see the below output:

[code]
Server up: http://localhost:3300
[/code]

Implementing GET /book

The below code implements this API:

[code language=”javascript”]
//Get all the books
app.get(‘/book’, function(req, res){
//Find all the books in the system.
Book.find({}, function(err, result){
if ( err ) throw err;
//Save the result into the response object.
res.json(result);
});
});
[/code]

In ExpressJS we create an application instance using express() (as seen above) and then setup the routes using the application instance. For example, for a GET call we use applicationInstance.get("ROUTE", handler(req, res){}), for a POST call we would set it up as applicationInstance.post("ROUTE", handler(req, res){}) and so on for other HTTP Methods like PUT, DELETE and others.

Implementing GET /book/:isbn

The below code is the implementation of the API

[code language=”javascript”]
//Get the details of the book with the given isbn
app.get(‘/book/:isbn’, function(req, res){
console.log("Fetching details for book with ISBN: " + req.params.isbn);
//The parameter in the route is accessed via request.params object.
Book.findOne({isbn: req.params.isbn}, function(err, result){
if ( err ) throw err;
res.json(result);
});
});
[/code]

In this Implementation the following are points of observation:

  1. Creating parameterized routes by using the :variable_Name in the route definition. In this case I have defined the route as /book/:isbn and this matches any url of form /book/abc123
  2. Accessing the path variable in the route via the request.params object. In this case I retrieve the ISBN string passed in the URL via req.params.isbn
  3. Using findOne with query object. In this case I am making use of MongoDB’s findOne() API along with the query object which is built using the ISBN information passed to retrieve one book information matching the ISBN

Implementing POST /book

This API is used to add a new book to the system. The information of the book is to be passed in the request body and we make use of the body-parser module to parse the json in the request body and populate it in the Node’s request object. The below code implements this API

[code language=”javascript”]
//Add a new book
app.post("/book", function(req, res){
console.log("Adding new Book: " + req.body.name);
var book = new Book({
name:req.body.name,
isbn: req.body.isbn,
author: req.body.author,
pages: req.body.pages
});

//Saving the model instance to the DB
book.save(function(err, result){
if ( err ) throw err;
//After successfully saving the book we generate a JSON response with the
//message and the inserted book information.
res.json({
messaage:"Successfully added book",
book:result
});
});
});
[/code]

In this implementation we read the JSON data passed by the Client, populate our instance of Book model class and persist the same to the DB using Mongoose API.

Implementation PUT /book/:isbn

This implementation is similar to GET /book/:isbn in terms of parameter handling and POST /book in terms of handling the book data passed from the client. We query for the existing information in the DB and then populate this existing data with the data passed from the client.

[code language=”javascript”]
//Update an existing book
app.put("/book/:isbn", function(req, res){
Book.findOne({isbn: req.params.isbn}, function(err, result){
if ( err ) throw err;

if(!result){
res.json({
message:"Book with ISBN: " + req.params.isbn+" not found.",
});
}

result.name = req.body.name;
result.isbn = req.body.isbn;
result.author = req.body.author;
result.pages = req.body.pages;

result.save(function(err, result){
if ( err ) throw err;
res.json({
message:"Successfully updated the book",
book: result
});
});

});
});
[/code]

Implementing DELETE /book/:isbn

Again this implementation is similar to GET /book/:isbn in terms of parameter handling and finding the book from the DB. To delete the book information we make use of findAndRemove API in Mongoose.

[code language=”javascript”]
//Delete an existing book
app.delete("/book/:isbn", function(req, res){
Book.findOneAndRemove({isbn: req.params.isbn}, function(err, result){
res.json({
message: "Successfully deleted the book",
book: result
});
});
});
[/code]

Testing the REST APIs

Now that I have shown you the implementation of individual APIs, lets go ahead and test the implemented APIs. For testing POST, PUT, DELETE I am making use of Postman Client’s plugin for Chrome.

GET /book

GET /book

GET /book/:isbn

GET /book/:isbn

POST /book

POST

PUT /book/:isbn

PUT

DELETE /book/:isbn

DELETE

Wrapping Up

I have shown you how the REST API is designed and then implemented using ExpressJS, Node.js and MongooseJS. For those looking for complete solution, copy the below code into a file named: express_demo.js

[code language=”javascript”]
//Saved in file: express_demo.js
//Express is required for creating Node.js based web apps
var express = require(‘express’);

//body-parser is used to parse the Request body and populate the req.
var bodyParser = require(‘body-parser’);

//mongoose is used for interacting with MongoDB
var mongoose = require(‘mongoose’);

var app = express();
app.set(‘port’, 3300);

//Configuring Express App to make use of BodyParser’s JSON parser to parse
//JSON request body
app.use(bodyParser.json());

var dbHost = ‘mongodb://localhost:27017/test’;
mongoose.connect(dbHost);
//Create a schema for Book
var bookSchema = mongoose.Schema({
name: String,
//Also creating index on field isbn
isbn: {type: String, index: true},
author: String,
pages: Number
});

//Create a Model by using the schema defined above
//Optionally one can provide the name of collection where the instances
//of this model get stored. In this case it is "mongoose_demo". Skipping
//this value defaults the name of the collection to plural of model name i.e books.
var Book = mongoose.model(‘Book’, bookSchema);

//Connecting to Mongod instance.
mongoose.connection;

//Starting up the server on the port: 3300
app.listen(app.get(‘port’), function(){
console.log(‘Server up: http://localhost:’ + app.get(‘port’));
});

//Get the details of the book with the given isbn
app.get(‘/book/:isbn’, function(req, res){
console.log("Fetching details for book with ISBN: " + req.params.isbn);

Book.findOne({isbn: req.params.isbn}, function(err, result){
if ( err ) throw err;
res.json(result);
});
});

//Get all the books
app.get(‘/book’, function(req, res){
//Find all the books in the system.
Book.find({}, function(err, result){
if ( err ) throw err;
//Save the result into the response object.
res.json(result);
});
});

//Add a new book
app.post("/book", function(req, res){
console.log("Adding new Book: " + req.body.name);
var book = new Book({
name:req.body.name,
isbn: req.body.isbn,
author: req.body.author,
pages: req.body.pages
});

//Saving the model instance to the DB
book.save(function(err, result){
if ( err ) throw err;
res.json({
messaage:"Successfully added book",
book:result
});
});
});

//Update an existing book
app.put("/book/:isbn", function(req, res){
Book.findOne({isbn: req.params.isbn}, function(err, result){
if ( err ) throw err;

if(!result){
res.json({
message:"Book with ISBN: " + req.params.isbn+" not found.",
});
}

result.name = req.body.name;
result.isbn = req.body.isbn;
result.author = req.body.author;
result.pages = req.body.pages;

result.save(function(err, result){
if ( err ) throw err;
res.json({
message:"Successfully updated the book",
book: result
});
});

});
});

//Delete an existing book
app.delete("/book/:isbn", function(req, res){
Book.findOneAndRemove({isbn: req.params.isbn}, function(err, result){
res.json({
message: "Successfully deleted the book",
book: result
});
});
});
[/code]

Before running this you would have to install the following packages: express, body-parser, mongoose using the command npm install.

Reference Books for Node.js

  • Professional Node.js: Building Javascript Based Scalable Software (English)
  • MongoDB : The Definitive Guide (English) 2nd Edition
  • MongoDB in Action (English)

Filed Under: ExpressJS, MongoDB, NodeJS Tagged With: NodeJS Tutorials

Mongoose – Node.js + MongoDB with Mongoose Tutorial

May 7, 2015 by Mohamed Sanaulla Leave a Comment

Develope Node.js + MongoDB Application with Mongoose

In my previous post, I demonstrated how to perform CRUD (Create, Read, Update, Delete) operations using Node.js and MongoDB API. This tutorial explains how to easily develop Node.js and MongoDB application with Mongoose application by using the Mongoose API. This tutorial assumes that reader has the knowledge on Node.js and MongoDB and knows how to set up the environment for this tutorial. Those who want to read the basic tutorials on Node.js and MongoDB please refer Introduction to Node.js and How to Install MongoDB on Windows.

If you are interested in receiving more updates on Node.js and MongoDB tutorials, please subscribe here.

Here is the overview of the sections covered in this tutorial.

  1. Introduction to Mongoose API
  2. Connecting to MongoDB using Mongoose
  3. Create Schema and Model in Mongoose
  4. Inserting Data using Mongoose
  5. Querying the DB using Mongoose
  6. Updating existing documents
  7. Deleting existing documents
  8. Example Application

Mongoose API

Mongoose is an mongodb object modelling API for node.js. Mongoose provides a straight-forward, schema-based solution to modeling your application data and includes built-in type casting, validation, query building, business logic hooks and more, out of the box. (Introduction taken from the Mongoose API Home page)

Mongoose ODMTo use mongoose API in node.js application one would have to download the Node package for mongoose API using the following command: npm install mongoose and then import the mongoose API package in your node.js application by using the following command: var mongoose = require('mongoose');

Connecting to MongoDB using Mongoose

The below code connects to the MongoDB database named test running on localhost, port 27017.

[code language=”javascript”]
var dbHost = ‘mongodb://localhost:27017/test’;
mongoose.connect(dbHost);
[/code]

After you have connected to the DB, you need to get a connection object. The connection object initially is in pending state and once it goes into open state we can proceed to interact with MongoDB. For this to happen we register a callback to listen to the open event, the below code registers to listen to error and open events fired by the connection.

[code language=”javascript”]
db.on(‘error’, console.error.bind(console, ‘connection error:’));
db.once(‘open’, function(){
console.log("Connected to DB");
//do operations which involve interacting with DB.
});
[/code]

Create Schema and Model in Mongoose

We need to create the schema of the document we are going to persist in the DB. While creating the schema we can mention the attributes of the document, the type of the attribute, additional information like whether an index needs to be created for the attribute and so on. In our example our book document would have the following attributes: name, isbn, author, pages. The below is the sample Schema I created for this example:

[code language=”javascript”]
//Create a schema for Book
var bookSchema = mongoose.Schema({
name: String,
//Also creating index on field isbn
isbn: {type: String, index: true},
author: String,
pages: Number
});
[/code]

Using the above schema we create a Model. The API to create model accepts the following: name of the model, name of the schema it has to use to create model class, name of the collection and whether to skip initialization. The below is the code to create Book Model from the schema:

[code language=”javascript”]
//Create a Model by using the schema defined above
//Optionally one can provide the name of collection where the instances
//of this model get stored. In this case it is "mongoose_demo". Skipping
//this value defaults the name of the collection to plural of model name i.e books.
var Book = mongoose.model(‘Book’, bookSchema, "mongoose_demo");
[/code]

Lets now see how to insert, read, update and delete records using the Model created above. Unlike previous post, in this post I will show you few examples of inserting records, querying for records, editing records and deleting records.

Inserting data using Mongoose

We create an instance of the Model class and then invoke save method on the instance to save the data to the DB. While invoking the save method we also provide a callback which gets executed after the save method returns either with an error or after successfully inserting the data. The below code creates two instances of Book model class and saves it to the DB.

[code language=”javascript”]
//Instantiating the Model – An instance of Model represents a mongodb document
var book1 = new Book({
name:"Mongoose Demo 1",
isbn: "MNG123",
author: "Author1, Author2",
pages: 123
});

//Saving the model instance to the DB
book1.save(function(err){
if ( err ) throw err;
console.log("Book Saved Successfully");
});

var book2 = new Book({
name:"Mongoose Demo 2",
isbn: "MNG124",
author: "Author2, Author3",
pages: 90
});

book2.save(function(err){
if ( err ) throw err;
console.log("Book Saved Successfully");
});
[/code]

Querying the DB using Mongoose

There are multiple APIs which one can use to query the collection in mongoDB, one can find all those APIs here. In this example I use find API to query the collection. The below code defines a function which queries for documents having less than 100 pages:

[code language=”javascript”]

var queryBooks = function(){
//Now querying those books which have less than 100 pages
//Find API takes in a query condition, attributes in the document to be projected,
//callback to be executed when the data is available.
Book.find({pages : {$lt:100}}, "name isbn author pages", function(err, result){
if ( err ) throw err;
console.log("Find Operations: " + result);
});
}
[/code]

Updating existing documents

Document can be updated by using the update API as shown below:

[code language=”javascript”]
var updateBook = function(){
/*
Find the book to be updated using the condition and then execute the update
passed to the API as the second argument
*/
Book.update({isbn : {$eq: ‘MNG125’}}, {$set: {name: "Mongoose Demo 3.1"}}, function(err, result){
console.log("Updated successfully");
console.log(result);
});
}

[/code]

Deleting existing documents

Documents can be deleted from the collection by using the remove API and by specifying the condition for selecting the documents to be deleted.

[code language=”javascript”]
var deleteBook = function(){
/*
When callback is not passed, the action is not invoked on the collection
until the exec() method is called.
In this case I am not passing the callback and instead executing the action
by invoking the exec() method.
*/
Book.remove({isbn:{$eq: ‘MNG124’}}).exec();
}
[/code]

Node.js + MongoDB + Mongoose Example Application

With this I have shown how the CRUD operations can be performed individually. Putting this all together into a working example given below:

[code language=”javascript”]
var mongoose = require(‘mongoose’);
var dbHost = ‘mongodb://localhost:27017/test’;
mongoose.connect(dbHost);
//Create a schema for Book
var bookSchema = mongoose.Schema({
name: String,
//Also creating index on field isbn
isbn: {type: String, index: true},
author: String,
pages: Number
});

//Create a Model by using the schema defined above
//Optionally one can provide the name of collection where the instances
//of this model get stored. In this case it is "mongoose_demo". Skipping
//this value defaults the name of the collection to plural of model name i.e books.
var Book = mongoose.model(‘Book’, bookSchema, "mongoose_demo");

var db = mongoose.connection;

db.on(‘error’, console.error.bind(console, ‘connection error:’));
db.once(‘open’, function(){
console.log("Connected to DB");

//Instantiating the Model – An instance of Model represents a mongodb document
var book1 = new Book({
name:"Mongoose Demo 1",
isbn: "MNG123",
author: "Author1, Author2",
pages: 123
});

//Saving the model instance to the DB
book1.save(function(err){
if ( err ) throw err;
console.log("Book Saved Successfully");
});

var book2 = new Book({
name:"Mongoose Demo 2",
isbn: "MNG124",
author: "Author2, Author3",
pages: 90
});

book2.save(function(err){
if ( err ) throw err;
console.log("Book Saved Successfully");
deleteBook();
});

var book3 = new Book({
name:"Mongoose Demo 3",
isbn: "MNG125",
author: "Author2, Author4",
pages: 80
});

book3.save(function(err){
if ( err ) throw err;
console.log("Book Saved Successfully");
queryBooks();
updateBook();

});

});

var queryBooks = function(){
//Now querying those books which have less than 100 pages
//Find API takes in a query condition, attributes in the document to be projected,
//callback to be executed when the data is available.
Book.find({pages : {$lt:100}}, "name isbn author pages", function(err, result){
if ( err ) throw err;
console.log("Find Operations: " + result);
});
}

var updateBook = function(){
/*
Find the book to be updated using the condition and then execute the update
passed to the API as the second argument
*/
Book.update({isbn : {$eq: ‘MNG125’}}, {$set: {name: "Mongoose Demo 3.1"}}, function(err, result){
console.log("Updated successfully");
console.log(result);
});
}

var deleteBook = function(){
/*
When callback is not passed, the action is not invoked on the collection
until the exec() method is called.
In this case I am not passing the callback and instead executing the action
by invoking the exec() method.
*/
Book.remove({isbn:{$eq: ‘MNG124’}}).exec();
}

[/code]

And querying for data from the mongo console we get the following:

[code language=”shell”]
> db.mongoose_demo.find()
{ "_id" : ObjectId("55490d87e34c8acc20b5363d"), "name" : "Mongoose Demo 1", "isbn" : "MNG123", "author" : "Author1, Author2", "pages" : 123, "__v" : 0 }
{ "_id" : ObjectId("55490d87e34c8acc20b5363f"), "name" : "Mongoose Demo 3.1", "isbn" : "MNG125", "author" : "Author2, Author4", "pages" : 80, "__v" : 0 }
[/code]

Summary

I hope that this tutorial would be very much helpful to understand how Mongoose API can be used for connecting to the MongoDB. It is obvious that with the help of Mongoose Object Data Mapping (ODM) once can write more complex CRUD operations with minimal effort. Please try the above examples write your feedback in the comments section.

Filed Under: MongoDB, NodeJS Tagged With: Mongoose, NodeJS Tutorials

Node.js + MongoDB – Performing CRUD Operations

April 26, 2015 by Mohamed Sanaulla Leave a Comment

In this post we will look at how we can implement CRUD operations using Node.js and MongoDB as the database. CRUD stands for Create Read Update Delete. The programming language we would use for this is Javascript based Node.js (Read : Introduction to Node.js) framework and MongoDB as the NoSQL database. And this example is going to be command line based and hence we will be taking user input from command line. If you have any questions, please write it in the comments section or post it in our facebook page.

NodeJS and MongoDB

To demonstrate CRUD operations I am picking a common example of storing, retrieving, updating and delete a book from the DB. The book has the following attributes: name, isbn, authors, page count. At the end of this post, I have given the complete example code for Node.JS and MongoDB integration example. In my next article, I will write about using Mongoose to facilitate the database modeling with Node.js.

also read:

  • How to Install Bower?
  • PrimeFaces + Spring Data + MongoDB Integration

This post is divided into following parts:

  1. Reading user input from command line in Node.js
  2. Connecting to MongoDB in Node.js
  3. Inserting record into MongoDB in Node.js
  4. Listing/Reading records from MongoDB in Node.js
  5. Updating records in MongoDB in Mode.js
  6. Deleting records from MongoDB in Node.js
  7. Node.js and MongoDB Integration Example

Reading user input from command line in Node.js

Node.js organizes the code into different Node packages ( think of them as packages in Java) and we can use those packages/modules by using the require function. On similar lines, there is a module in Node.js called readline which provides APIs to read from different input streams. The below code demonstrates how one can read from command line in Node.js

[code language=”javascript”]
//input.js

//Importing the relevant module
var readline = require(‘readline’);

//Initializing the reading interface to read from
//stdin and write to stdout
var rl = readline.createInterface({
input: process.stdin,
output: process.stdout
});

//Asking for user input and providing a callback to deal with the user input
rl.question("What is your name? ", function(answer){
console.log("Your name is: " + answer);
console.log("Press Ctrl+C to end the program");
});

[/code]

The output would be:

[code]
G:\node\testapp>node input.js
What is your name? Mohamed
Your name is: Mohamed
Press Ctrl+C to end the program
[/code]

Connecting to MongoDB in Node.js

  • Read : How to Install MongoDB on Windows

Lets now see how to connect to MongoDB from Node.js. As with reading user input, for MongoDB as well we have a Node.js module called mongodb. But this module would have to be downloaded using NPM (Node Package Manager). The mongodb module provides the driver to connect to MongoDB DB from Javascript. To install the mongodb module using NPM run the following command from your command prompt: npm install mongodb.

Note: NPM comes with the Node.js installation and you need not do anything additional to install NPM.

The below code shows how to connect to MongoDB and fetch records from a collection which already had few records populated. Getting familiarity with the MongoDB commands is required. You can find a very detailed documentation on MongoDB here.

[code language=”javascript”]
//FileName: mongodb.js

//Importing the required mongodb driver
var MongoClient = require(‘mongodb’).MongoClient;

//MongoDB connection URL
var dbHost = ‘mongodb://localhost:27017/test’;

//Name of the collection
var myCollection = "people";

//Connecting to the Mongodb instance.
//Make sure your mongodb daemon mongod is running on port 27017 on localhost
MongoClient.connect(dbHost, function(err, db){
if ( err ) throw err;
//Query Mongodb and iterate through the results
db.collection(myCollection).find({},{},{}).toArray(
function(err, docs){
for(index in docs){
console.log(docs[index]);
}
}
);
});

[/code]

Now that we have seen how to accept user input and how to connect to MongoDB, I will next show how to perform CRUD operations using our Book example.

CRUD Operations

We would take the data to be inserted into the DB via the command line and also the choice of operation to be performed i.e whether to Create OR Read OR Update OR Delete. The below code shows how the menu is printed and accepts the user choice for the operation.

[code language=”javascript”]
var readline = require(‘readline’);
var rl = readline.createInterface({
input: process.stdin,
output: process.stdout
});

//The below code prints the menu
var printMenu = function(db){
console.log("Welcome to CRUD demo using Node.js and MongoDB");
console.log("1. Insert a new Book");
console.log("2. List all the books");
console.log("3. Update the book by ISBN");
console.log("4. Delete the book by ISBN");
console.log("5. Quit");
rl.question("Enter your choice: ", function(answer){
console.log("Choice entered is: " + answer);
switch(answer){
case "1":
insertBook(dbConn);
break;
case "2":
listBooks(dbConn);
break;
case "3":
updateBook(dbConn);
break;
case "4":
deleteBook(dbConn);
break;
case "5":
console.log("Press Ctrl+C to exit the program");
return;
}

});
}
[/code]

Inserting record into MongoDB in Node.js

Inserting a new record involves taking user input i.e name of the book, isbn of the book, authors and page count. After this we use the MongoDB API to execute the insert statement. The below code is the method to insert a new document.

[code language=”javascript”]
var insertBook = function(db){
rl.question("Enter the name of the book: ", function(bookName){
rl.question("Enter the ISBN of the book: ", function(isbn){
rl.question("Enter the authors of the book[Comma separated if more than 1]: ", function(author){
rl.question("Enter the total number of pages: ", function(pageCount){
db.collection(myCollection).find({isbn: isbn},{},{}).toArray(
function(err, docs){
if ( docs.length > 0){
console.log("Book with ISBN " + isbn + " already exists");
printMenu(dbConn);
}else{
db.collection(myCollection).insert({
‘name’:bookName,
‘isbn’: isbn,
‘author’: author,
‘pages’: pageCount
}, bookInsertHandler);
}
}
);
});
});
});
});
}
[/code]

Listing/Reading records from MongoDB in Node.js

The MongoDB API provides find method to retrieve the results from the DB. The find method accepts three parameters namely: the query clause, the project clause i.e which attributes from the collection to be retrieved and the sort clause. The below code fetches and displays the result:

[code language=”javascript”]
var listBooks = function(db){
db.collection(myCollection).find({},{},{}).toArray(
function(err, docs){
for(index in docs){
console.log(docs[index]);
}
printMenu(dbConn);
}
);
}
[/code]

Updating records in MongoDB in Node.js

To update the book we need to take ISBN number to retrieve the book and update its data. Also we take user input for the newer data and accordingly use the MongoDB API to update the record. The below code updates the record:

[code language=”javascript”]
var updateBook = function(db){
rl.question("Enter the ISBN of the book you want to update: ", function(answer){

db.collection(myCollection).find({isbn: answer},{},{}).toArray(
function(err, docs){
if ( docs.length == 0){
console.log("Book with ISBN " + isbn + " not found");
printMenu(dbConn);
}else{
rl.question("Enter the name of the book: ", function(bookName){
rl.question("Enter the authors of the book[Comma separated if more than 1]: ", function(author){
rl.question("Enter the total number of pages: ", function(pageCount){
db.collection(myCollection).update({"isbn":answer}, {
‘name’:bookName,
‘author’: author,
‘pages’: pageCount,
‘isbn’:answer
}, bookUpdateHandler);
});
});
});
}
});
});
}
[/code]

Deleting records from MongoDB in Node.js

To delete the record we take the ISBN number for the book and use the MongoDB API to delete the record. The below code does the operation.

[code language=”javascript”]
var deleteBook = function(db){
rl.question("Enter the ISBN of the book you want to update: ", function(answer){
db.collection(myCollection).find({isbn: answer},{},{}).toArray(
function(err, docs){
if ( docs.length == 0){
console.log("Book with ISBN " + answer + " not found");
printMenu(dbConn);
}else{
db.collection(myCollection).remove({"isbn":answer}, bookDeleteHandler);
}
});
});
}
[/code]

Node.js + MongoDB Example

The complete sample program to demonstrate CRUD operations in Nodde.js and MongoDB integration example is given below.

[code language=”javascript”]
//Filename: crud.js
var readline = require(‘readline’);
var rl = readline.createInterface({
input: process.stdin,
output: process.stdout
});

var MongoClient = require(‘mongodb’).MongoClient;
var dbHost = ‘mongodb://localhost:27017/test’;
var myCollection = "crud";
var dbConn;

var bookInsertHandler = function(err, recs){
if(err) throw err;
console.log("Successfully inserted the book into database");
printMenu(dbConn);
}

var bookUpdateHandler = function(err, recs){
if(err) throw err;
console.log("Successfully updated the book");
printMenu(dbConn);
}
var bookDeleteHandler = function(err, recs){
if(err) throw err;
console.log("Successfully deleted the book");
printMenu(dbConn);
}
var printMenu = function(db){
console.log("Welcome to CRUD demo using Node.js and MongoDB");
console.log("1. Insert a new Book");
console.log("2. List all the books");
console.log("3. Update the book by ISBN");
console.log("4. Delete the book by ISBN");
console.log("5. Quit");
rl.question("Enter your choice: ", function(answer){
console.log("Choice entered is: " + answer);
switch(answer){
case "1":
insertBook(dbConn);
break;
case "2":
listBooks(dbConn);
break;
case "3":
updateBook(dbConn);
break;
case "4":
deleteBook(dbConn);
break;
case "5":
console.log("Press Ctrl+C to exit the program");
return;
}

})
}

var insertBook = function(db){
rl.question("Enter the name of the book: ", function(bookName){
rl.question("Enter the ISBN of the book: ", function(isbn){
rl.question("Enter the authors of the book[Comma separated if more than 1]: ", function(author){
rl.question("Enter the total number of pages: ", function(pageCount){
db.collection(myCollection).find({isbn: isbn},{},{}).toArray(
function(err, docs){
if ( docs.length > 0){
console.log("Book with ISBN " + isbn + " already exists");
printMenu(dbConn);
}else{
db.collection(myCollection).insert({
‘name’:bookName,
‘isbn’: isbn,
‘author’: author,
‘pages’: pageCount
}, bookInsertHandler);
}
}
);
});
});
});
});
}

var listBooks = function(db){
db.collection(myCollection).find({},{},{}).toArray(
function(err, docs){
for(index in docs){
console.log(docs[index]);
}
printMenu(dbConn);
}
);
}

var updateBook = function(db){
rl.question("Enter the ISBN of the book you want to update: ", function(answer){

db.collection(myCollection).find({isbn: answer},{},{}).toArray(
function(err, docs){
if ( docs.length == 0){
console.log("Book with ISBN " + isbn + " not found");
printMenu(dbConn);
}else{
rl.question("Enter the name of the book: ", function(bookName){
rl.question("Enter the authors of the book[Comma separated if more than 1]: ", function(author){
rl.question("Enter the total number of pages: ", function(pageCount){
db.collection(myCollection).update({"isbn":answer}, {
‘name’:bookName,
‘author’: author,
‘pages’: pageCount,
‘isbn’:answer
}, bookUpdateHandler);
});
});
});
}
});
});
}

var deleteBook = function(db){
rl.question("Enter the ISBN of the book you want to update: ", function(answer){
db.collection(myCollection).find({isbn: answer},{},{}).toArray(
function(err, docs){
if ( docs.length == 0){
console.log("Book with ISBN " + answer + " not found");
printMenu(dbConn);
}else{
db.collection(myCollection).remove({"isbn":answer}, bookDeleteHandler);
}
});
});
}

MongoClient.connect(dbHost, function(err, db){
if ( err ) throw err;
dbConn = db;
printMenu();
});
[/code]

To run the above code use the following command: node crud.js

Summary

  • Node.js Tutorials
  • MongoDB Tutorials

This tutorial walks you through how to write a simple CRUD operations with the combination of Node.js and MongoDB technologies. These two are using JSON as the data format that makes it to fit together perfectly. I hope you would have got good idea  on how to create, read, update and delete a record in MongoDB using Node.js script. If you have any questions, please write it in the comments section. In my next article, I will write about MongoDB with Mongoose API.

Filed Under: MongoDB, NodeJS Tagged With: MongoDB Tutorials, NodeJS Tutorials

Jongo – The Java Object Document Mapper for MongoDB

May 24, 2014 by Mohammad Umar Ali Karimi Leave a Comment

This tutorial explains about the Jongo, a Java query language for accessing the data from MongoDB database. When you work with the MongoDB database, you have to use the Mongo shell for querying the data, however when you are querying in the Java application, you have to use the Java driver for MongoDB and retrieve the data. This approach is not easy and needs complex syntax. Jongo addresses the problem by Query in Java as in Mongo shell.

  • How to install MongoDB on Windows

MongoDB is an open-source noSQL database written in C++ that provides high performance, high availability and auto-scaling. A record in a MongoDB is called a Document. A document is a data structure composed of keys and corresponding values. MongoDB documents are similar to JSON objects as shown:

MongoDB record

MongoDB document

Java MongoDB Driver

MongoDB provides a Java driver to communicate with MongoDB database. Please follow  ttp://docs.mongodb.org/ecosystem/drivers/java for more details on how to connect to a MongoDB server via Java application.

DBObject API

The Java driver for MongoDB uses DBObject interface and its subclasses for persisting and representing documents in the database. This API provides bare bones methods to persist and retrieve Java objects into MongoDB database.

Below is the class diagram of DBObject API

DBObjectAPI
Lets use the DBObject to persist a Java POJO into MongoDB.

Person.java

[code lang=”java”]
public class Person {
private String _id;
private String firstName;
private String lastName;
private Address address;
//getters & setters
}
[/code]

Address.java

[code lang=”java”]
public class Address {
private String houseNo;
private String street;
private String city;
private String country;
private String zip;
}
[/code]

There are two POJOs Address and Person and Address is referenced inside Person. We’ll first try to insert an instance of the Person as Mongodb Document. Also, let’s create a Mongo Persistence DAO and its corresponding implementation.

MongoService.java

[code lang=”java”]
public interface MongoService<T> {
Object save (T t);
T get(String key);
List<T> getAll();
void dropCollection(String collectionName);
}
[/code]

MongoServiceImpl.java

[code lang=”java”]
public class MongoServiceImpl implements MongoService<Person> {
protected String connectionURL = "localhost:27017";
public static final String collectionName = "persons";
protected DBCollectiom collection;
protcted DB;

public MongoService() {
db = MongoConnectionManager.connect(connectionURL);
collection = db.getCollection(collectionName);
}
//….
}
[/code]

Below is the implementation of save() method:

[code lang=”java”]
public Object save(Person p) {
//first extract Address from person and build the corresponding BasicDBObject
BasicDBObject addrObj = new BasicDBObject();
Address addr = p.getAddress();
addrObj.append("houseNo", addr.getHouseNo());
addrObj.append("street", addr.getStreet());
addrObj.append("city", addr.getCity());
//….
//Now build the BasicDBObject for Person
BasicDBObject personObj = new BasicDBObject();
personObj.append("firstName", t.getFirstName());
personObj.append("lastName", t.getLastName());
personObj.append("address", addrObj); //Note how addrObj is mapped here

//finally save the personObj to the MongoDB database
collection.insert(personObj);
return personObj.get("_id");
}
[/code]

To save a Person instance having an embedded Address object we have to create two BasicDBObject instances and use hard coded keys to populate them. A call to insert(DBObject o) method of DBCollection class to store the person instance in the database. Finally, we simply retrieve the persisted document id using the get(String key) method of BasicDBObject and passed "_id" to retrieve it. The retrieved id can be used to query the persisted object.

Let’s retrieve the persisted MongoDB document from the database by querying the peristed id.

[code lang=”java”]
public Person get(String id) {
BasicDBObject persistent = (BasicDBObject)collection.findOne(new ObjectId(id));
//The retrieved object is a BasicDBObject. Extract values from this object and
//build a person object.
Person person = new Person();
Set<String> keySet = persistent.keySet();
for(String key: keySet) {
Object value = persistent.get(key);
buildPerson(key, value, person);
}
return person;
}
[/code]

Limitations

Its evident from the above example that saving and retrieving a document using the DBObject API is very daunting. We can easily lose track of POJO attributes that serves as keys to the MongoDB document. For any project that has a number of POJOs, saving and retrieving objects using DBObject API will be very frustrating. Java Mongo Driver is a poor man’s tool for performing database operations.

Using JSON Jackson along with Mongo Java Driver – The Jongo Way

Using bare bone DBObject API can be frustrating. However, a combination of JSON Jackson and Mongo Java Driver can ease the storage and retrieval of POJOs. Moreover, the embedded POJOs don’t have to be dealt separately while persisting and retrieval. The following piece of code makes use of byte[] to marshall and unmarshall Java objects.

[code language=”java”]
public Object save(Object o) throws IOException {
ObjectWriter writer = mapper.writer();
ByteArrayOutputStream baos = new ByteArrayOutputStream();
writer.writeValue(baos, o);
DBObject dbObject = new LazyWriteableDBObject(baos.toByteArray(), new LazyBSONCallback());
DBObject result = new BasicDBObject();
result.putAll(dbObject);
collection.insert(result);
return result.get("_id");
}
[/code]

The save(Object o) method use ByteArrayOutputStream to create a DBObject. The DBObject is then persisted. In this case, we don’t have to create separate DBObject instances for Container POJO and embedded POJO. The complete object graph is first converted to ByteArrayOutputStream and then persisted.

[code language=”java”]
public static <T> T getPojo(DBObject o, Class<T> clazz) throws IOException {
ObjectReader reader = mapper.reader(clazz);
DBEncoder dbEncoder = DefaultDBEncoder.FACTORY.create();
OutputBuffer buffer = new BasicOutputBuffer();
dbEncoder.writeObject(buffer, o);
T pojo = reader.readValue(buffer.toByteArray());
return pojo;
}
[/code]

The getPojo() method converts a given DBObject to the required class type. The JSON Jackson ObjectReader converts a given DBObject to the POJO type.

Jongo makes use of the Jackson JSON API and Mongo Driver. Instead of converting results into strings and unmarshalling strings into objects, Jongo use byte[] for query marshalling and unmarshalling. Jongo also provides Aggregation f/w to work with MongoDB Aggregations. More on Jongo…

Filed Under: MongoDB Tagged With: Jongo

Enable MongoDB Journalling

April 25, 2014 by Krishna Srinivasan Leave a Comment

MongoDB uses an on disk journal to guarantee write operations and to provide crash resiliency. MongoDB creates a journal for every write that includes the exact disk location and the bytes that changed in the write. Thus if you have a crash in the server, the journal can be used to replay any writes that have not yet been written to the data files.

MongoDB uses memory mapped files to write your data to disk. By default MongoDB data files are flushed to disk every 60 s. MongoDB also uses memory mapped files for the journal. By default the journal is flushed to disk every 100 ms. Since the final data files are flushed to disk every 60s the journal does not need to track writes for more than 60s. For more details of the mechanics of the journalling refer to the official documentation.

Journalling Write Concern

[code]
>db.data.insert({"name":"testentry"});
>db.runCommand({"getLastError":1, "j":true});
[/code]

When you turn on journalling you also have to ability to specify a write concern of ‘Journalled’ for your MongoDB operations. This implies that mongod confirms the write operation only after committing to the journal. However this has a drawback – when you specify “j”:true with getLastError MongoD will wait about 1/3 of the journalcommitinternal before committing the journal data. The default journalcommitinterval is 100ms – so mongod will wait 30ms and commit the data. This essentially means that on a single thread you can only get about 33.3 writes/sec. The recommended practice is to batch your writes. For example if you have 50 writes use the “j”:true setting only on the last write – this will acknowledge that all the previous 50 writes are done.

Summary

Every production MongoDB instance should run with journalling enabled. If you dont have journalling enabled and your server or mongod process crashes MongoDB will not be ensure data integrity. You will need to run a “repair” operation on the database which depending on the amount of data could take a few hours to complete. Turn it off only if you really know what you are doing. At MongoDirector all our instances follow the mongodb best practice configuration and have journalling turned on by default.

This article is originally published at Mongo Director Blog, re-posted here with authors permission and as part of the JBC program.

Filed Under: MongoDB Tagged With: MongoDB Journal

10 Tips To Improve MongoDB Security

April 23, 2014 by Krishna Srinivasan Leave a Comment

MongoDB provides a number of constructs to improve the security of your data. The security of your data in MongoDB is paramount – so it is important to leverage these constructs to reduce your surface area. Here are 10 tips you can use to improve the security of your MongoDB servers on premise and in the cloud.

1. Enable auth

Even if you have deployed your Mongodb servers in a trusted network it is good security practice to enable auth. It provides you “Defense in depth” if your network is compromised. Edit your mongod configuration file to enable auth.

[code]auth = true[/code]

2. Don’t expose your production db to the internet

Restricting physical access to your database is an important aspect of security. If it is not necessary do not expose your production database to the internet. In case of any compromise if an attacker cannot physically connect to your MongoDB server, your data is that much more secure. If you are on AWS you can place your db’s in a VPC private subnet.

3. Use firewalls

Use firewalls to restrict which other entities are allowed to connect to your mongodb server. Best practice is to only allow your application servers access to the database. If you are hosted on AWS use ‘Security groups’ to restrict access. If you are hosted on a provider that does not support firewall constructs you can easily configure it yourself using ‘iptables’. Refer to the mongodb documentation to configure iptables for your scenario.

4. Use key files to setup the replica set

Specify a shared key file to enable communication between your mongodb instances in a replica set. To enable this add the keyfile parameter to the config file as below. The contents of the file need to be the same on all the machines.

[code]keyFile = /srv/mongodb/keyfile[/code]

5. Disable HTTP status interface

Mongodb by default provides a http interface running by default on port 28017 which provides the “home” status page. This interface is not recommended for production use and is best disabled. Use the “nohttpinterface” configuration setting to disable the http interface.

[code]nohttpinterface = true[/code]

6. Disable the REST interface

The monogdb REST interface is not recommended for production. It does not support any authentication. It is turned off by default. If you have turned it on using the “rest” configuration option you should turn it off for production systems.

[code]rest = false[/code]

7. Configure Bind_ip

If your system has multiple network interfaces you can use the “bind_ip” option to restrict your mongodb server to listen only on the interfaces that are relevant. By default mongodb will bind to all the interfaces

[code]bind_ip = 10.10.0.25,10.10.0.26[/code]

8. Enable SSL

If you don’t use SSL your data is traveling between your Mongo client and Mongo server unencrypted and is susceptible to eavesdropping, tampering and “man in the middle” attacks. This is especially important if you are connecting to your Mongodb server over unsecure networks like the internet.

9. Role based authorization

MongoDB supports role based authentication to give you fine grained control over the actions that can be performed by each user. Use role based constructs to restrict access instead of making all your users admins. Refer to the roles documentation for more details.

10. Enterprise mongodb & Kerberos

Enterprise mongodb integrates with Kerberos for authentication. Refer to the mongodb documentation for more details. Username/password systems are inherently insecure – use kerb based authentication if possible.

This article is originally published at Mongo Director Blog, re-posted here with authors permission and as part of the JBC program.

Filed Under: MongoDB Tagged With: MongoDB Security

Best MongoDB GUI Tools

April 23, 2014 by Krishna Srinivasan Leave a Comment

A good UI is an important part of the development experience. The mongo shell works great for administrative actions but when working with larger amounts of data, the UI becomes fairly important. There are a number of options when it comes to MongoDB GUI, some good and some not so good. Below are some of the options for MongoDB UI we have considered. Our main scenarios are data visualization, presentation and editing. As always your mileage might vary depending on your scenario and preferences.

1. MongoVue

MongoVue is a desktop GUI for the windows platform. It has a simple clean UI and the basic features are free. Data can be presented in text view, tree view or table view. You can also save your find queries for later usage – we find this rather convenient. The more advanced features requires you to purchase a license. Overall the software appears stable and well maintained. mongovue gui tool

2. MongoHub

MongoHub is a native Mac GUI for MongoDB. It provides you an option to connect to your MongoDB server through a SSH tunnel which is fairly convenient from a security perspective. The “StatMonitor” feature give you a live display of stats a.k.a mongostat. The query interface is a little bit more limited in that it only seems to support a tree view. Also there is no way to save a find query for later. In our experience the software appears to work fairly well but it doesn’t seem like it is being maintained – so use at your own risk. mongohub gui tool

3. RockMongo

RockMongo is a html based MongoDB GUI. The GUI is authored in PHP and is open source. The downside of the html based approach is that you need a PHP server to run this GUI. You can also choose to run the PHP server on your local box. The UI is no frill, fairly easy to use and supports all the common options of working with collections, stats etc. The find interface only presents data in a tabular/text model – so it could be an issue when you are working with multilevel documents. Also there doesn’t seem to be a lot of checkin activity – so we suspect the project is inactive. rockmongo gui tool

4. RoboMongo

RoboMongo is shell centric MongoDB GUI that supports the Windows, MacOS and linux platforms. It’s still early days for RoboMongo with the latest version being 0.8.4. It is also one of the few GUI’s that supports SSL connections to your mongodb server. There is also support for connecting through a SSH tunnel. The query interface displays data in tree view, table view and text view. You can also save your queries for later usage. One of the coolest features is that it also has support for the shell – so you can still continue to use the shell commands that you are comfortable with. There are some quirks in the UI which I think will get worked out over time. If actively maintained I think this could be the best cross platform GUI for MongoDB. robomongo

This article is originally published at Mongo Director Blog, re-posted here with authors permission and as part of the JBC program.

Filed Under: MongoDB Tagged With: MongoDB Tools

How To Install MongoDB On Windows

April 21, 2014 by Amr Mohammed Leave a Comment

MongoDB, is an open source document database and the leading NO SQL database written in C++. MongoDB is used as a persistence store and it get adapted by different latest libraries and implementations, so Eclipse JPA/NoSQL has provided a library for get your application integrated with MongoDB as Spring Data. In this tutorial, we are going to install and configure the MonogoDB step-by-step.

Download MonogDB Installation File

To download the proper installation file for your operating system you have to follow the following link. For this tutorial we are going to install the MongoDB for Windows.

MongoDB Required Installation File Windows

Install MongoDB

After downloading the required file, you have to install it on your operating system, by double clicking on the installed file, it will start the installation process. Follow the below steps for finishing the installation process.

MongoDB Installation First Step

  • By pass the welcome screen by clicking on the next.

MongoDB Installation Second Step

  • Accept the terms in the License Agreement

MongoDB Installation Third Step

  • Use a Custom installation for more controlling upon where is the location of the installation and what’s the component that are being installed.

MongoDB Installation Forth Step

  • By clicking Browse, you have to change the location of the installation.
  • By expanding the root feature tree, you can select/deselect the required/un-required features.

MongoDB Installation Change Installation Path

  • The preferred location for installing MongoDB, is to install it using a path with no spaces. The suggested path for this guide is C:\mongodb\.

MongoDB enable all features

  • Enable all features.

Pre Install

  • Click install to begin the installation process

Installing Process

  • The installation in progress.

Installation has finished

  • MongoDB Installation is completed, If you see the above screen.

MongoDB Configurations

MongoDB isn’t traditional database, so you are not about using or dealing with Schemas and Database Tables, but you are about document based data store. The concept of document based isn’t like the well-known document that every one could use it, rather it’s JSON document, and it needs a location for being saved. For that purpose, you shall configure that location the documents are saved within it. Such a configuration is what you will be seeing by following the below steps:

  • Go into your installation directory and based on our tutorial, it’s C:\mongodb.
  • Create a folder for being considered as a repository location. For that you can create C:\mongodb\data\db.
  • That path should be passed through starting up the MongoDB instance.

Create Data Folder

  • This is the form of the directories after created the data location.

Data location After Starting Up MongoDB

  • MongoDB has used the created directory for bootstrapping and initializing the required data.

Starting Up and Connecting MongoDB

Once you are installing the configuring the MongoDB you are ready for starting it up and making it ready for use. Execute the following commands for a proper initialization of MongoDB.

  • Go into the installation directory, and based on our tutorial, it’s C:\mongodb.
  • Navigate into \bin.
  • Execute the mongodb.exe –dbpath=”C:\mongodb\data\bin” for passing the desired saving location that the MongoDB has to use for storing the supposed document that persisted by the client. Client here, is whatever application that want to connect the MongoDB, so it could be a user that connected MongoDB through the command line, or it could be an application that using the required library for connecting the MongoDB.
  • The successful execution for the above command, should configure the mongodb and it should also starting it up and it is  ready for use.

Starting MongoDB

  • Starting Up the MongoDB has by turn initialized the MongoDB and locating the passed location for the next coming further operations.

Connecting MongoDB

For connecting MongoDB, you have multiple choices:

  • Connecting MongoDB using the command line by navigating into C:\mongodb\bin and executing mongo.exe.
  • If you’ve not provided any name after mongo.exe, you are almost connecting the default database test.
  • If you’ve provided a name, the mongodb itself will try to locate that name as a database name, if it didn’t find that database, it will create it automatically.
  • MongoDB will create & open the database that you’ve passed through mongo.exe. In the below screenshot, we are creating a database called “JavaBeat”.

Connecting Database

  • By connecting MongoDB, you have the ability to operate and acting on the connected database.

Sample Find Operation in MongoDB

The below snapshot shows you a sample example of executing find operation upon USER collections (Collection is an alternative concept of Table).

Find operation

References

  • MongoDB
  • correspondences between SQL and Mongo
  • What is NOSQL?

Filed Under: MongoDB Tagged With: MongoDB Configurations

  • 1
  • 2
  • Next Page »

Follow Us

  • Facebook
  • Pinterest

As a participant in the Amazon Services LLC Associates Program, this site may earn from qualifying purchases. We may also earn commissions on purchases from other retail websites.

JavaBeat

FEATURED TUTORIALS

Answered: Using Java to Convert Int to String

What is new in Java 6.0 Collections API?

The Java 6.0 Compiler API

Copyright © by JavaBeat · All rights reserved