This GridFS tutorial explains how to use GridFS for stroring the larger files in MongoDB and when to use GridFS in MongoDB. This GridFS tutorial stores image file of 16.5 MB size to illustrate an example.
MongoDB limits the size of the document in a collection to 16 MB. Realistically this should be good enough for data which is a group of key-value pairs. But when you want to store files along with this key-value pairs, we often find the upper limit of 16 MB to be a limitation. To overcome this, MongoDB provides an API called GridFS where we can store files greater than 16 MB along with any metadata related to the file.
In this post we will look at how we can store and retrieve files from and to mongodb using GridFS and MongoDB java driver. Let us first look at the basics of GridFS.
GridFS Tutorial
Basics of GridFS
GridFS stores the files in the form of a series of chunks where by default each chunk size can be of maximum 255k. There are two collections namely files
and chunks
. The files
collection stores the metadata of the file and chunks
collection stores the chunks of the files with each chunk having file_id
and n
where file_id
is the _id
of the parent chunk and n
is the chunk number.
The collections files
and chunks
are stored under a namespace. By default the namespace is fs
. We can override the namespace and provide our own.
Saving files to Mongodb using GridFS
Let us first look at saving file to Mongodb using GridFS. For this I am going to consider an image with size 16.5MB.
- Also Read : Node.js + MongoDB – Performing CRUD Operations
Note: You can download any image file for your use which is large in size.
The below code saves the image at a given location to Mongodb using GridFS:
import java.io.File; import java.io.IOException; import com.mongodb.DB; import com.mongodb.DBCollection; import com.mongodb.MongoClient; import com.mongodb.gridfs.GridFS; import com.mongodb.gridfs.GridFSInputFile; public class GridFsSaveDemo { public static void main(String[] args) throws IOException { MongoClient mongo = new MongoClient("localhost", 27017); DB db = mongo.getDB("filesdb"); //Location of file to be saved String imageLocation = "C:/Users/Mohamed/Pictures/image2.jpg"; //Create instance of GridFS implementation GridFS gridFs = new GridFS(db); //Create a file entry for the image file GridFSInputFile gridFsInputFile = gridFs.createFile(new File(imageLocation)); //Set a name on GridFS entry gridFsInputFile.setFilename("image1"); //Save the file to MongoDB gridFsInputFile.save(); } }
Let us open mongo
shell to verify if the above file got saved:
> use filesdb switched to db filesdb > show collections fs.chunks fs.files system.indexes > db.fs.chunks.find().count() 67 > db.fs.files.find().pretty() { "_id" : ObjectId("55e3279f355311259428f3a9"), "filename" : "image1", "aliases" : null, "chunkSize" : NumberLong(261120), "uploadDate" : ISODate("2015-08-30T15:56:15.017Z"), "length" : NumberLong(17315594), "contentType" : null, "md5" : "a6bc0171f7beff8036715dd1d022c1a0" }
You can notice above that the file was divided into 67 chunks stored in fs.chunks
collection and file metadata i.e the fix size, file name, upload date stored in fs.files
collection.
Reading files from MongoDB using GridFS
The code for reading the image saved above is given below:
import java.io.IOException; import com.mongodb.DB; import com.mongodb.MongoClient; import com.mongodb.gridfs.GridFS; import com.mongodb.gridfs.GridFSDBFile; public class GridFsReadDemo { public static void main(String[] args) throws IOException { MongoClient mongo = new MongoClient("localhost", 27017); DB db = mongo.getDB("filesdb"); //Create instance of GridFS implementation GridFS gridFs = new GridFS(db); //Find the image with the name image1 using GridFS API GridFSDBFile outputImageFile = gridFs.findOne("image1"); //Get the number of chunks System.out.println("Total Chunks: " + outputImageFile.numChunks()); //Location of the image read from MongoDB to be written String imageLocation = "C:/Users/Mohamed/Pictures/mongoImage.jpg"; outputImageFile.writeTo(imageLocation); mongo.close(); } }
The above code queries MongoDB to find the image chunks by using the image name. And then writes those image chunks to file system.
GridFS makes it very easy to store and retrieve files from MongoDB. We can use it to store not only files greater then 16MB but also store files lesser than 16MB.
I hope this GridFS tutorial helped you to understand how to store the files larger than 16 MB and storing an image file in the MongoDB database.