Tuning FotoWeb and Index Manager for performance
Introduction
Users expect a site to load in two seconds or less, and research shows that users tend to abandon a site that isn't loaded within three seconds. Google has carried out extensive testing on user behavior relating to page load times and the results show that users easily navigate away from a site if it becomes less responsive.
With this in mind, system administrators in a FotoWare DAM system must be aware of hardware requirements, configuration, and regular maintenance tasks, and they must know how to configure the system.
FotoWare system - hardware requirements
Different types of applications have different requirements: Some are CPU-intensive, some are memory-intensive, and others are highly dependent on I/O. A FotoWare system requires all three.
The following table explains why.
| Resource | Why? |
|---|---|
| CPU | Calculations, especially rendition and transcoding tasks. |
| Memory |
Image and video subsampling (large files, large calculations). Caching of immediate objects. Keeping database indexes readily available. |
| IO |
Network bandwidth - Speed of delivery to clients and the number of concurrent users. Disk bandwidth - time to load and serve data (renditions, metadata, and so on) and speed of the database. |
Reducing latency
Latency is the time interval that passes from when a user performs an operation until they see the response of that action. It is not to be confused with interface sluggishness, although such sluggishness can be a symptom of high network latency. Bandwidth and latency are not always connected; a high-bandwidth network may be able to push enormous amounts of data quickly, but with considerable latency, a system built on such a network can still be perceived as slow by users for its sheer response time.
Key points for reducing latency
- Faster networks
- Is fiber an option?
- In extreme cases, shorter network cables?
- Reduction of middleware
- Can firewalls, proxies, routers, and/or switches be bypassed? Such components inevitably induce a level of network latency. With only a few connector points, this may not be noticeable, but in large network environments, they can have a marked impact on performance.
- Faster CPUs
- SSD Drives
- Prefer unencrypted over encrypted protocols
A real-life FotoWeb performance example
Disclaimer: The test results below are meant as an example only, and reflect the performance of this specific system. Actual performance relies on several factors, including, but not limited to, the system's hardware and software configuration.
Using the Apache benchmarking tool , FotoWare has run a real-life performance test to measure the delivery speed of uncached and cached content from FotoWeb.
TEST IMAGE
File size: 7552 Kb
Dimensions: 5184 x 3456 px (18 megapixels)
Resolution: 240.00 PPI
Benchmarking hardware: Lenovo T430 portable - Intel i7 2.9 GHz, 16GB memory, Dual SSD
Uncached loading results
The screenshot below shows the time it takes to load the uncached image (red text, select the image to open a larger version).
How does this scale?
Browsers typically open 8 connections to load content and users may have to wait when content is uncached.
| 1 Core | 4 Cores | 8 Cores | 16 Cores | |
|---|---|---|---|---|
| 1 User | 0.8 | 0.2 | 0.1 | 0.1 |
| 10 Users | 8 | 2 | 0.5 | 0.25 |
| 100 Users | 80 | 40 | 20 | 10 |
| 1000 Users | 800 | 400 | 200 | 100 |
Time is in seconds
Results when content is already cached
These are the results of cached content delivery using the same Apache benchmarking utility to deliver a single file (select the image to open a larger version).
How does that scale?
| 1 Core | 4 Cores | 8 Cores | 16 Cores | |
|---|---|---|---|---|
| 1 User | 0.024 | 0.006 | 0.003 | 0.0015 |
| 10 Users | 0.24 | 0.06 | 0.03 | 0.015 |
| 100 Users | 2.4 | 0.6 | 0.3 | 0.15 |
| 1000 Users | 24 | 6 | 3 | 1.5 |
Time is in seconds
What to do if there is lots of uncached content in the archives?
In certain environments, such as where large image feeds are received, there may be large portions of uncached content at any one time. To compensate for such content, the solution is to add more hardware:
- Faster disks
- Faster CPUs
- More CPUs
Another option is to add more Index Manager servers to help the system keep up with the incoming flow of files.
FotoWare 8.0 architecture - do's and don'ts
The chart below shows the system architecture in FotoWare version 8.0. Select the image to open a larger version.
The key point to note is that FotoWeb needs low-latency access to both MongoDB and Index Manager. They should be on the same LAN and not be constrained by any firewalls. Often a firewall can slow performance considerably and performance can vary a lot depending on a number of factors - at times it may work just fine while at other times performance may be sluggish. Because troubleshooting such issues can be a challenge, we recommend running these components on the same LAN and making sure there's minimal latency between components.
Strongly discouraged configurations
The chart below shows a system where a firewall filters communication between FotoWeb/MongoDB and Index Manager, and the storage system on the other. Index Manager relies on being able to push metadata to MongoDB (the FotoWeb metadata database) and introducing latency at that level can negatively affect performance. Also, the client communication from FotoWeb, which runs on port 7000/7001 by default, will also have to be funneled through the firewall, which can have a negative effect on how users perceive client performance.
Finally, the last chart shows a configuration where a firewall has been placed between FotoWeb and the MongoDB instance. This configuration should be avoided since it adds latency to so many of the core functions in FotoWeb that it is guaranteed to generate performance issues.
Recommendations
Ideally, any firewall on the system limits incoming traffic from the internet to the FotoWeb server, for instance, by allowing only ports 80 and/or 443. Performance-wise, it is best to place the entire system - FotoWeb, Index Manager, and file storage, in the DMZ zone so the servers can communicate freely without firewall restrictions.
Configuring Index Manager for use with FotoWeb
To reduce the possibility that users execute queries that generate a lot of server processing and/or extremely large results, you can remove the option to use wildcard characters at the beginning of the search string.
- In the Index Manager app, go to Indexes and open the Advanced tab for the relevant index.
- Select Disable leading search characters
For more information, see Disabling leading search characters.
Configuring FotoWeb for optimal searching
From the Tools menu (cogwheel icon) go to Site Configuration > Searching.
A system administrator can configure FotoWeb to only allow searches to return a certain number of hits. This prevents users from performing very wide queries that take a lot of time to process and also reduces the server traffic that's generated when users scroll through their results in the thumbnail grid.
Also, it's good practice to set as low a search timeout as possible so that if Index Manager fails to deliver a search result within the set time frame, you know that the performance problem is Index Manager related.
If the search timeout period is too long, the server's capacity to serve results to a large number of concurrent users will be reduced.
For more information about searching in FotoWeb, see Searching in FotoWeb.
Example: Index Manager performance benchmarking
| SATA Instance | SSD instance | |
|---|---|---|
| CPU | 8 vCPUs, AMD Opteron 2,1 GHz | 8 vCPUs, AMD Opteron 2,1 GHz |
| RAM | 30GB | 30GB |
| Disks | RAID 10-protected SATA | RAID 10-protected SSD |
| Index size | 100,475 JPEGs | 100,475 JPEGs |
Indexing speed
This table shows the time it took to index the roughly 100,000 files on the two systems. Cache warming is the process of creating cached low-resolution thumbnail representations of indexed assets that can be delivered to clients.
| SATA Instance | SSD instance | SSD Gain | |
|---|---|---|---|
| Indexing | 18m 50s | 11m 11s | -38% |
| Cache warming | 13m 20s | 5m 31s | -58% |
As the table shows, reading metadata (the indexing process) requires much IO, and generating thumbnails during cache warming is even more IO-intensive.
Query speed
For these tests the Index Manager cache was disabled to show actual disk IO performance measures.
| SATA Instance | SSD instance | SSD Gain | |
|---|---|---|---|
| Single result | 10 ms | 5 ms | -50% |
| ~4k results | 100 ms | 50 ms | -50% |
| ~40k results | 750 ms | 450 ms | -40% |
| ~100k results | 1800 ms | 1200 ms | -22% |
As the table shows, searching is also IO intensive. The servers were equally equipped apart from the storage system. From a user's perspective, anything below 100 ms will be perceived as instant. It should be noted that the times reported here are the search times reported by Index Manager, so any network latency will affect the time it takes before the resulting thumbnails actually show up on the client's screen.
Naturally, the size of the result set also affects the performance of the query, so educating users on how to create meaningful queries is important. Using taxonomies to assist users in filtering the archive content is also valuable in this regard.
Tools for analyzing Index Manager performance
Index Manager 8.0 has tools to measure its performance using Windows utilities. For more information, see Analyzing Index Manager performance.