Skip to main content

 

Documentation & User Guides | FotoWare

Tuning FotoWeb and Index Manager for performance

What's in this document?

According to surveys by Akamai and Gomez.com (http://blog.kissmetrics.com/loading-time) nearly half of web users expect a site to load in two seconds or less, and they tend to abaondon a site that isn't loaded within three seconds. Google has also done extensive testing on user behavior relative to page load times that show how easily users navigate away if a site becomes less responsive.

What this means in a FotoWare DAM system is that system administrators must be wary of hardware requirements, configuration, regular maintenance tasks and absolute do's and don'ts in terms of how the system is configured. This document aims to give some pointers on these matters.

What does a FotoWare system require, hardware-wise?

Different types of applications have different requirements: Some are CPU-intensive, some are memory-intensive and others are highly dependent on I/O.

A FotoWare system requires no less than all three. The table below explains why.

Resource Why?
CPU Calculations, especially rendition and transcoding tasks
Memory

Image and video subsampling (large files, large calculations).

Caching of immediate objects

Keeping database indexes readily available

IO

Network bandwidth - Speed of delivery to clients and number of concurrent users

Disk bandwidth - time to load and serve data (renditions, metadata etc) and speed of database

Reducing latency

Latency is the time interval that passes from a user performs an operation until he sees the response of that action. It is not to be confused with interface sluggishness, although such sluggishness can be a symptom of high network latency. Bandwidth and latency are not always connected; a high-bandwidth network may be able to push enormous amounts of data quickly, but with considerable latency a system built on such a network can still be perceived as slow by users for its sheer response time.

Thus, here are some key points for reducing latency.

  • Faster networks
    • Is fiber an option?
    • In extreme cases, shorter network cables?
  • Reduction of middleware
    • Can firewalls, proxies, routers and/or switches be bypassed? Such components  will inevitably induce a level of network latency. With only a few connector points, this may not be perceivable, but in large network environments, they can have a very notable effect on performance.
  • Faster CPUs
  • SSD Drives
  • Prefer unencrypted over encrypted protocols

A real-life FotoWeb performance example

Disclaimer: The test results below are meant as an example only, and reflects the performance of this specific system. Actual performance will rely on several factors, including but not limited to the system's hardware and software configuration.

FotoWare has made some real-life performance testing using the Apache benchmarking tool to measure the delivery speed of uncached and cached content from FotoWeb.

 

Apache performance testing example.png

TEST IMAGE

File size: 7552 Kb

Dimensions: 5184 x 3456 px (18 Megapixels)

Resolution: 240.00 ppi

Benchmarking hardware: Lenovo T430 portable - Intel i7 2.9 GHz, 16GB memory, Dual SSD

Uncached loading results

The below screenshot shows the time it takes to load the uncached image (emphasized in red; click for larger version).

Uncached_results.png

How does this scale?

Browsers typically open 8 connections to load content, and users may have to wait when content is uncached.

  1 Core 4 Cores 8 Cores 16 Cores
1 User 0.8 0.2 0.1 0.1
10 Users 8 2 0.5  0.25
100 Users 80 40 20 10
1000 Users 800 400 200 100

Time is in seconds

Results when content is already cached

These are the results of cached content delivery using the same Apache benchmarking utility to deliver a single file. (Click to view larger version)

Cached_results.png

How does that scale?

  1 Core 4 Cores 8 Cores 16 Cores
1 User 0.024 0.006 0.003 0.0015
10 Users 0.24 0.06 0.03 0.015
100 Users 2.4 0.6 0.3 0.15
1000 Users 24 6 3 1.5

Time is in seconds

 

What to do if there is lots of uncached content in the archives?

In certain environments, such as where large image feeds are received, there may be large portions of uncached content at any one time. To compensate for such content, the solution is to add more hardware:

  • Faster disks
  • Faster CPUs
  • More CPUs

 

Another option is to add more Index Manager servers to help the system more easily keep up with the incoming flow of files.

The FotoWare 8.0 architecture - do's and don'ts

The below chart show the system architecture in FotoWare version 8.0. Click for a larger version.

fw8 architecture.png

The key points to note here is that FotoWeb needs low-latency access to both MongoDB and Index Manager. They should be on the same LAN and not be constrained by any firewalls. Often a firewall can slow performance considerably and performance can vary a lot depending on a number of factors - at time it may work just fine, at other times performance may be sluggish. Because troubleshooting such issues can be a challenge, we recommend running these components on the same LAN and making sure there's minimal latency between components.

Strongly discouraged configurations

The below chart shows a system where a firewall filters communications between FotoWeb/MongoDB and Index Manager and the storage system on the other. Index Manager relies on being able to push metadata to MongoDB (the FotoWeb metadata database) and introcuing latency at that level can have a negative effect on performance. Also the client communication from FotoWeb, which runs on port 7000/7001 by default, will also have to be funneled through the firewall, which can have a negative effect on how users perceive client performance.

fw8 architecture - discouraged.png

 

Finally, the last chart shows a configuration where a firewall has been placed between FotoWeb and the MongoDB instance. This configuration should be avoided at all costs, since it adds latency to so many of the core functions in FotoWeb that it is positively guaranteed to generate performance issues.

fw8 architecture - very strongly discouraged.png

Recommendations

Ideally, any firewall on the system will limit incoming traffic from the Internet to the FotoWeb server, for instance by allowing only ports 80 and/or 443. Performance-wise, it is best to place the entire system - FotoWeb, Index Manager, and file storage, in the DMZ zone to allow the servers to communicate freely without firewall restrictions.

 

Configuring Index Manager for use with FotoWeb

IM search settings for FW.png

Setting a search threshold

When configuring Index Manager for use with a FotoWeb, it's good practice to reduce the maximum number of hits a search can produce by setting an abort threshold. The will cut off a search when it produces more than the preset number of hits and deliver that search result to the client. This is a good, practical measure to prevent bogging down the server with heavy queries that will produce more hits than a user can anyhow evaluate. Using search thresholds has some implications that you can read more about by following this link.

Remove users' possibility to search for *searchword

To reduce the possibility that users execute queries that generate a lot of server processing and/or extremely large results, it is possible to remove their possibility to use wildcards at the beginning of the search string. To do so, tick Disable leading search characters in the index configuration.

Configuring FotoWeb for optimal searching

FW settings for optimal searching.png

Reduce the maximum number of hits

A system administrator can configure FotoWeb to only allow searches to return a certain number of hits. This prevents users from performing very wide queries that take a lot of time to process and also reduces the server traffic that's generated when users scroll through their results in the thumbnail grid.

Also, its good practice to set as low a search timeout as possible. That way, if Index Manager fails to deliver a search result within the set time frame, you know that the performance problem is Index Manager related. 

Note that if the search timeout period is too long, the server's capacity to serve results to a large number of concurrent users will be reduced.

Example: Index Manager performance benchmarking

  SATA Instance SSD instance
CPU 8 vCPUs, AMD Opteron 2,1 GHz 8 vCPUs, AMD Opteron 2,1 GHz
RAM 30GB 30GB
Disks RAID 10-protected SATA RAID 10-protected SSD
Index size 100,475 JPEGs 100,475 JPEGs

Indexing speed

This table shows the time it took to index the roughly 100,000 files on the two systems. Cache warming is the process of creating cached lowres thumbnail representations of indexed assets that can be delivered to clients.

  SATA Instance SSD instance SSD Gain
Indexing 18m 50s 11m 11s -38%
Cache warming 13m 20s 5m 31s -58%

As the table shows, reading metadata (the indexing process) requires much IO, and generating thumbnails during cache warming is even more IO-intensive.

Query speed

For these tests the Index Manager cache was disabled to show actual disk IO performance measures.

  SATA Instance SSD instance SSD Gain
Single result 10 ms 5 ms -50%
~4k results 100 ms 50 ms -50%
~40k results 750 ms 450 ms -40%
~100k results 1800 ms 1200 ms -22%

As the table shows, searching is also IO intensive. The servers were equally equipped apart from the storage system. From a user's perspective anything below 100 ms will be perceived as instant. It should be noted that the times reported here are the search times reported by Index Manager, so any network latency will affect the time it takes before the resulting thumbnails actually show up on the client's screen.

Naturally, the size of the result set alo affects the performance of the query, so educating users on how to create meaningful queries is important. Using taxonomies to assist users in filtering the archive content is also valuable in this regard.

Tools for analyzing Index Manager performance

Index Manager 8.0 has tools that make it possible to measure its performance using Windows utilities. Learn more about using Index Manager performance counters by following this link.