This GridFS tutorial explains how to use GridFS for stroring the larger files in MongoDB and when to use GridFS in MongoDB. This GridFS tutorial stores image file of 16.5 MB size to illustrate an example.
MongoDB limits the size of the document in a collection to 16 MB. Realistically this should be good enough for data which is a group of key-value pairs. But when you want to store files along with this key-value pairs, we often find the upper limit of 16 MB to be a limitation. To overcome this, MongoDB provides an API called GridFS where we can store files greater than 16 MB along with any metadata related to the file.
In this post we will look at how we can store and retrieve files from and to mongodb using GridFS and MongoDB java driver. Let us first look at the basics of GridFS.
GridFS Tutorial
Basics of GridFS
GridFS stores the files in the form of a series of chunks where by default each chunk size can be of maximum 255k. There are two collections namely files and chunks. The files collection stores the metadata of the file and chunks collection stores the chunks of the files with each chunk having file_id and n where file_id is the _id of the parent chunk and n is the chunk number.
The collections files and chunks are stored under a namespace. By default the namespace is fs. We can override the namespace and provide our own.
Saving files to Mongodb using GridFS
Let us first look at saving file to Mongodb using GridFS. For this I am going to consider an image with size 16.5MB.
- Also Read : Node.js + MongoDB – Performing CRUD Operations
Note: You can download any image file for your use which is large in size.
The below code saves the image at a given location to Mongodb using GridFS:
[code lang=”java”]
import java.io.File;
import java.io.IOException;
import com.mongodb.DB;
import com.mongodb.DBCollection;
import com.mongodb.MongoClient;
import com.mongodb.gridfs.GridFS;
import com.mongodb.gridfs.GridFSInputFile;
public class GridFsSaveDemo {
public static void main(String[] args) throws IOException {
MongoClient mongo = new MongoClient("localhost", 27017);
DB db = mongo.getDB("filesdb");
//Location of file to be saved
String imageLocation = "C:/Users/Mohamed/Pictures/image2.jpg";
//Create instance of GridFS implementation
GridFS gridFs = new GridFS(db);
//Create a file entry for the image file
GridFSInputFile gridFsInputFile = gridFs.createFile(new File(imageLocation));
//Set a name on GridFS entry
gridFsInputFile.setFilename("image1");
//Save the file to MongoDB
gridFsInputFile.save();
}
}
[/code]
Let us open mongo shell to verify if the above file got saved:
[code lang=”shell”]
> use filesdb
switched to db filesdb
> show collections
fs.chunks
fs.files
system.indexes
> db.fs.chunks.find().count()
67
> db.fs.files.find().pretty()
{
"_id" : ObjectId("55e3279f355311259428f3a9"),
"filename" : "image1",
"aliases" : null,
"chunkSize" : NumberLong(261120),
"uploadDate" : ISODate("2015-08-30T15:56:15.017Z"),
"length" : NumberLong(17315594),
"contentType" : null,
"md5" : "a6bc0171f7beff8036715dd1d022c1a0"
}
[/code]
You can notice above that the file was divided into 67 chunks stored in fs.chunks collection and file metadata i.e the fix size, file name, upload date stored in fs.files collection.
Reading files from MongoDB using GridFS
The code for reading the image saved above is given below:
[code lang=”java”]
import java.io.IOException;
import com.mongodb.DB;
import com.mongodb.MongoClient;
import com.mongodb.gridfs.GridFS;
import com.mongodb.gridfs.GridFSDBFile;
public class GridFsReadDemo {
public static void main(String[] args) throws IOException {
MongoClient mongo = new MongoClient("localhost", 27017);
DB db = mongo.getDB("filesdb");
//Create instance of GridFS implementation
GridFS gridFs = new GridFS(db);
//Find the image with the name image1 using GridFS API
GridFSDBFile outputImageFile = gridFs.findOne("image1");
//Get the number of chunks
System.out.println("Total Chunks: " + outputImageFile.numChunks());
//Location of the image read from MongoDB to be written
String imageLocation = "C:/Users/Mohamed/Pictures/mongoImage.jpg";
outputImageFile.writeTo(imageLocation);
mongo.close();
}
}
[/code]
The above code queries MongoDB to find the image chunks by using the image name. And then writes those image chunks to file system.
GridFS makes it very easy to store and retrieve files from MongoDB. We can use it to store not only files greater then 16MB but also store files lesser than 16MB.
I hope this GridFS tutorial helped you to understand how to store the files larger than 16 MB and storing an image file in the MongoDB database.
Leave a Reply