Skip to main content
Documentation & User Guides | Fotoware

Tuning FotoWeb and Index Manager for performance

Introduction

Users expect a site to load in two seconds or less, and research shows that users tend to abandon a site that isn't loaded within three seconds. Google has carried out extensive testing on user behavior relating to page load times and the results show that users easily navigate away from a site if it becomes less responsive.

With this in mind, system administrators in a Fotoware DAM system must be aware of hardware requirements, configuration, and regular maintenance tasks, and they must know how to configure the system. 

Fotoware system - hardware requirements

Different types of applications have different requirements: Some are CPU-intensive, some are memory-intensive, and others are highly dependent on I/O. A Fotoware system requires all three.

The following table explains why.

Resource Why?
CPU Calculations, especially rendition and transcoding tasks.
Memory

Image and video subsampling (large files, large calculations).

Caching of immediate objects.

Keeping database indexes readily available.

IO

Network bandwidth - Speed of delivery to clients and the number of concurrent users.

Disk bandwidth - time to load and serve data (renditions, metadata, and so on) and speed of the database.

Reducing latency

Latency is the time interval that passes from when a user performs an operation until they see the response of that action. It is not to be confused with interface sluggishness, although such sluggishness can be a symptom of high network latency. Bandwidth and latency are not always connected; a high-bandwidth network may be able to push enormous amounts of data quickly, but with considerable latency, a system built on such a network can still be perceived as slow by users for its sheer response time.

Key points for reducing latency

  • Faster networks
    • Is fiber an option?
    • In extreme cases, shorter network cables?
  • Reduction of middleware
    • Can firewalls, proxies, routers, and/or switches be bypassed? Such components inevitably induce a level of network latency. With only a few connector points, this may not be noticeable, but in large network environments, they can have a marked impact on performance.
  • Faster CPUs
  • SSD Drives
  • Prefer unencrypted over encrypted protocols

A real-life FotoWeb performance example

DisclaimerThe test results below are meant as an example only, and reflect the performance of this specific system. Actual performance relies on several factors, including, but not limited to, the system's hardware and software configuration.

Using the Apache benchmarking tool , Fotoware has run a real-life performance test to measure the delivery speed of uncached and cached content from FotoWeb.

 

Apache performance testing example.png

TEST IMAGE

File size: 7552 Kb

Dimensions: 5184 x 3456 px (18 megapixels)

Resolution: 240.00 PPI

Benchmarking hardware: Lenovo T430 portable - Intel i7 2.9 GHz, 16GB memory, Dual SSD

Uncached loading results

The screenshot below shows the time it takes to load the uncached image (red text, select the image to open a larger version).

Uncached_results.png

How does this scale?

Browsers typically open 8 connections to load content and users may have to wait when content is uncached.

  1 Core 4 Cores 8 Cores 16 Cores
1 User 0.8 0.2 0.1 0.1
10 Users 8 2 0.5  0.25
100 Users 80 40 20 10
1000 Users 800 400 200 100

Time is in seconds

Results when content is already cached

These are the results of cached content delivery using the same Apache benchmarking utility to deliver a single file (select the image to open a larger version).

Cached_results.png

How does that scale?

  1 Core 4 Cores 8 Cores 16 Cores
1 User 0.024 0.006 0.003 0.0015
10 Users 0.24 0.06 0.03 0.015
100 Users 2.4 0.6 0.3 0.15
1000 Users 24 6 3 1.5

Time is in seconds

 

What to do if there is lots of uncached content in the archives?

In certain environments, such as where large image feeds are received, there may be large portions of uncached content at any one time. To compensate for such content, the solution is to add more hardware:

  • Faster disks
  • Faster CPUs
  • More CPUs

Another option is to add more Index Manager servers to help the system keep up with the incoming flow of files.

Fotoware 8.0 architecture - do's and don'ts

The chart below shows the system architecture in Fotoware version 8.0. Select the image to open a larger version.

fw8 architecture.png

The key point to note is that FotoWeb needs low-latency access to both MongoDB and Index Manager. They should be on the same LAN and not be constrained by any firewalls. Often a firewall can slow performance considerably and performance can vary a lot depending on a number of factors - at times it may work just fine while at other times performance may be sluggish. Because troubleshooting such issues can be a challenge, we recommend running these components on the same LAN and making sure there's minimal latency between components.

Strongly discouraged configurations

The chart below shows a system where a firewall filters communication between FotoWeb/MongoDB and Index Manager, and the storage system on the other. Index Manager relies on being able to push metadata to MongoDB (the FotoWeb metadata database) and introducing latency at that level can negatively affect performance. Also, the client communication from FotoWeb, which runs on port 7000/7001 by default, will also have to be funneled through the firewall, which can have a negative effect on how users perceive client performance.

fw8 architecture - discouraged.png

 

Finally, the last chart shows a configuration where a firewall has been placed between FotoWeb and the MongoDB instance. This configuration should be avoided since it adds latency to so many of the core functions in FotoWeb that it is guaranteed to generate performance issues.

fw8 architecture - very strongly discouraged.png

Recommendations

Ideally, any firewall on the system limits incoming traffic from the internet to the FotoWeb server, for instance, by allowing only ports 80 and/or 443. Performance-wise, it is best to place the entire system - FotoWeb, Index Manager, and file storage, in the DMZ zone so the servers can communicate freely without firewall restrictions.

 

Configuring Index Manager for use with FotoWeb

To reduce the possibility that users execute queries that generate a lot of server processing and/or extremely large results, you can remove the option to use wildcard characters at the beginning of the search string.

IM search settings for FW.png

  1. In the Index Manager app, go to Indexes and open the Advanced tab for the relevant index.
  2. Select Disable leading search characters

 For more information, see Disabling leading search characters.

Configuring FotoWeb for optimal searching

From the Tools menu (cogwheel icon) go to Site Configuration Searching.

FW settings for optimal searching.png

A system administrator can configure FotoWeb to only allow searches to return a certain number of hits. This prevents users from performing very wide queries that take a lot of time to process and also reduces the server traffic that's generated when users scroll through their results in the thumbnail grid.

Also, it's good practice to set as low a search timeout as possible so that if Index Manager fails to deliver a search result within the set time frame, you know that the performance problem is Index Manager related. 

If the search timeout period is too long, the server's capacity to serve results to a large number of concurrent users will be reduced.

For more information about searching in FotoWeb, see Searching in FotoWeb.

Example: Index Manager performance benchmarking

  SATA Instance SSD instance
CPU 8 vCPUs, AMD Opteron 2,1 GHz 8 vCPUs, AMD Opteron 2,1 GHz
RAM 30GB 30GB
Disks RAID 10-protected SATA RAID 10-protected SSD
Index size 100,475 JPEGs 100,475 JPEGs

Indexing speed

This table shows the time it took to index the roughly 100,000 files on the two systems. Cache warming is the process of creating cached low-resolution thumbnail representations of indexed assets that can be delivered to clients.

  SATA Instance SSD instance SSD Gain
Indexing 18m 50s 11m 11s -38%
Cache warming 13m 20s 5m 31s -58%

As the table shows, reading metadata (the indexing process) requires much IO, and generating thumbnails during cache warming is even more IO-intensive.

Query speed

For these tests the Index Manager cache was disabled to show actual disk IO performance measures.

  SATA Instance SSD instance SSD Gain
Single result 10 ms 5 ms -50%
~4k results 100 ms 50 ms -50%
~40k results 750 ms 450 ms -40%
~100k results 1800 ms 1200 ms -22%

As the table shows, searching is also IO intensive. The servers were equally equipped apart from the storage system. From a user's perspective, anything below 100 ms will be perceived as instant. It should be noted that the times reported here are the search times reported by Index Manager, so any network latency will affect the time it takes before the resulting thumbnails actually show up on the client's screen.

Naturally, the size of the result set also affects the performance of the query, so educating users on how to create meaningful queries is important. Using taxonomies to assist users in filtering the archive content is also valuable in this regard.

Tools for analyzing Index Manager performance

Index Manager 8.0 has tools to measure its performance using Windows utilities. For more information, see Analyzing Index Manager performance.

  • Was this article helpful?