Auto-tagging
Auto-tagging
In FotoWare, auto-tagging refers to the automated process of assigning descriptive tags to images.
Using Artificial Intelligence (AI), FotoWare can automatically detect keywords, brands, and faces and extract text content from images. FotoWare uses Azure Cognitive Services for AI. For more information, see Cognitive Services—APIs for AI Solutions | Microsoft Azure. Auto-tagging uses a set of features supported by Computer Vision in Azure Cognitive Services.
FotoWare sends images to Azure Cognitive Services by using either an action or an asset webhook, and the tagging results will be saved to selected metadata fields on the asset in FotoWare. Auto-tagging is only available for image files; the file sent to Azure Cognitive Services is always a jpeg.
In FotoWare, auto-tagging is configured as an action or asset-ingested webhook. Both use the same URL format to call auto-tagging, and the configuration settings are provided as part of the URL in both cases. For more information on configuration, see Configuring auto-tagging. Since auto-tagging results are machine generated, we recommend they are reviewed by a person and, as such, recommend tagging images on demand.
Features
Customers can configure which of the features they would like to use, and they can mix and match features as necessary. The following features are supported:
- Description
Auto-tagging can analyze an image and generate a human-readable phrase that describes its content. The algorithm returns several descriptions based on different visual features, and each description is given a confidence score. The final output is a list of descriptions ordered from highest to lowest confidence. Azure Computer Vision only returns descriptions in English, but we use Azure Translator services to translate the description to all languages.
Image Analysis can return content tags for thousands of recognizable objects, living beings, scenery, and actions that appear in images. Tagging is not limited to the main subject, such as a person in the foreground, it also includes the setting (indoor or outdoor), furniture, tools, plants, animals, accessories, gadgets, and so on.
- Tags
Image Analysis can return content tags for thousands of recognizable objects, living beings, scenery, and actions that appear in images. Tagging is not limited to the main subject, such as a person in the foreground, it also includes the setting (indoor or outdoor), furniture, tools, plants, animals, accessories, gadgets, and so on.
- Brands/Logos
Brand detection uses a database of thousands of global logos to identify commercial brands in images. The Computer Vision service detects whether there are brand logos in a given image; if there are, it returns the brand name. Note that this feature identifies company logos in images. Brand names, for instance, will not be identified but could be picked up by OCR.
- People
Image Analysis can detect if there are human faces within an image and return a true/false value.
- Optical Character Recognition (OCR)
OCR is also referred to as text recognition or text extraction. With machine-learning-based OCR techniques, you can extract printed or handwritten text from images, such as posters, street signs, and product labels. The text is typically extracted as words, text lines, and paragraphs or text blocks, enabling access to the digital version of the scanned text. This eliminates or significantly reduces the need for manual data entry.
Languages supported
Azure Cognitive Services - Computer Vision has different language support for different features. For an overview of supported languages, see Azure Cognitive Services - Computer Vision - Language support.
To help support the same multiple languages for all features in auto-tagging in FotoWare, we use the Azure translation service to translate texts from English for the languages not natively supported in Computer Vision. This extends the language set supported, but auto-translation does not always guarantee 100% accuracy.