Skip to main content

 

Documentation & User Guides | FotoWare

MongoDB disk and memory requirements

Disk requirements

The amount of disk space required by MongoDB depends entirely on the number of assets in the system. If the number of assets is stable, or if there is a steady influx / deletion rate in the archive so that the number of assets remains at a certain level, the instance will not grow.

Each asset remains approximately 10 days in MongoDB after deletion. Hence, the approximate formula for calculating the number of assets can be expressed as follows:

 

N = i * (t + 10) + p

p: Number of assets that are permanently in the system

t: Max number of days an asset is kept in the system after ingestion

i: Number of assets ingested per day (24h)

N: Total number of assets in the system at any given time.

 

Each asset requires approximately 10Kb of space. The formula to calculate the disk space required for assets in MongoDB goes like this:

Disk space (Megabytes) = N * 0,01

(N is the total number of assets, as explained above.)

Disk performance is important

MongoDB data should be stored on a fast, local SSD for the best performance. Storing MongoDB data on a network drive will not work.

All indexed files are pushed to MongoDB

When an Index Manager is configured to push data to a FotoWeb server's MongoDB instance, all the indexed content from that Index Manager server is pushed to MongoDB, even if only some of the indexes are used as archives in FotoWeb. For example, there may be a case where FotoWeb hosts only a few archives from the Index Manager server while other indexes are used by FotoStation. In this case, data from all indexes is pushed to the MongoDB server, since Index Manager is considered a "slave" of the FotoWeb server.

MongoDB cleanup routine

MongoDB does not free up disk space when data is deleted. However, already allocated disk space can be reused by new data. There are MongoDB admin commands that can be used to shrink and defragment the database files. However, FotoWeb does not automatically run such commands on MongoDB.

FotoWeb  uses the “smallfiles” option of MongoDB to ensure that disk space is allocated in small chunks rather than huge blocks with exponentially increasing sizes. Older versions of FotoWeb did not do this, so older installations may have large MongoDB data files which are mostly unused.

Important: Never delete the database files

MongoDB is NOT a cache - It is a database. Valuable data such as albums, CMS exports and (starting with Feature Release 4) users and groups are stored in MongoDB.

Deleting database files causes loss of data that can only be restored from a backup.

Storing the MongoDB database files in an alternative location

The location of the MongoDB data files is not configurable per se. However, you can create a file system junction point to place the folder elsewhere.

Memory requirements

MongoDB requires approximately 1GB of RAM per 100.000 assets. If the system has to start swapping memory to disk, this will have a severely negative impact on performance, and should be avoided.

Page File requirements

Also, it's important to follow MongoDB's outlined requirements for Page File size. The following information is copied from the MongoDB Production Notes:

Configure the page file such that the minimum and maximum page file size are equal and at least 32 GB. Use a multiple of this size if, during peak usage, you expect concurrent writes to many databases or collections. However, the page file size does not need to exceed the maximum size of the database.

A large page file is needed as Windows requires enough space to accommodate all regions of memory mapped files made writable during peak usage, regardless of whether writes actually occur.

The page file is not used for database storage and will not receive writes during normal MongoDB operation. As such, the page file will not affect performance, but it must exist and be large enough to accommodate Windows’ commitment rules during peak database use.

NOTE: Dynamic page file sizing is too slow to accommodate the rapidly fluctuating commit charge of an active MongoDB deployment. This can result in transient overcommitment situations that may lead to abrupt server shutdown with a VirtualProtect error 1455.