All accounts in Mediagraph, including free trial accounts, get access to auto tagging for no additional charge.
There’s a lot of buzz around auto tagging through Artificial Intelligence/Machine Learning. But in many cases it fails to live up to the hype. Worse yet, you may not find out about it until after you have spent a lot of time and money on implementation. Or maybe you commit to a service that is not particularly good for your needs. Wouldn’t it be great to try it out before making a major commitment? MediaGraph makes this very easy (and free) to do.
Currently, we offer three types of auto tagging - auto transcription for videos, auto keyword tagging, and sensitive content detection. MediaGraph lets you take our auto tagging for a spin, even before you have paid for an account. You can upload thousands of files, and see what Amazon Rekognition AI can do. Depending on the content, your needs, and the existing tags, it might be really helpful, kind of helpful, or just meh.
Let’s take a look.
Set up a trial account
If you don’t already have an account, set up a 30 day trial account for free - no credit card needed. Just go to this page, enter your email address, and start the account. (If you already have an account, skip to the next step).
Enable Auto Tagging
Go to the Manage tab, and open the Site Settings tab (this is only available to account Admins and Owners). Click “Enable Auto Tagging.” We suggest you leave the threshold at 80% and leave “Show General Members Auto Tags” unchecked. Settings should look like this.
You can add auto tagging to any account in the Site Settings.
Upload test files or Auto tag existing files
Once you have enabled auto tagging, all newly uploaded images will get the auto tags. So just upload your test files, and give it a little time to run the tagging in the background. It’s useful to upload a broad variety of files in order to see what kind of information the auto tagging can extract.
Note that a trial account can hold up to 10GB. This means you can upload about 8000 JPEG files in the demo account. (The 8000 count is based on a 3000 pixel JPEG file, saved a #8 quality in photoshop - about 1.2 megabytes each). That should be enough to understand what the tagging can do.
Upload files and they will be auto tagged.
Tagging existing files
If you already have an account with images, you’ll need one extra step to tag these existing files. Select images you wish to auto tag, and click the action menu. In the Add Metadata flyout, you’ll see Run Auto Tagging. Check that and click submit. (You won't need to do this for files uploaded in the future.)
Note that auto tagging can take a little while to complete, especially for large groups of images. This is an operation that runs in the background. You can monitor the progress by looking at the Auto Tagging filter panel. It will show you how many files have been processed.
Manually submit previously uploaded files using the Action Menu.
Review the tags
Once the tagging has completed, you can take a look. Open the Filter panel and find the Auto Tagging panel. When you open it, you’ll see a list of the 50 most common tags for the files currently showing. Review these to see the results. You can click on any of the tags to filter down to just those items. You are likely to see a mixed bag of results - some good and some bad.
If you have uploaded a lot of files, the 50 tag cutoff will hide a lot of the tags that are made. To better evaluate, you may want to look at smaller groups of files, such as those in an individual folder or collection.
Which files were missed?
You can also see which files did not get a particular tag. This can tell you what the auto tagging missed. When you hover over the tag in the filter panel, you have the option to choose “does not have tag”. This will hide all files that have the tag, and leave untagged files showing. By browsing through the files, you can see if the auto tagging has missed files.
You can toggle between "has tag" and "does not have tag" to see how thoroughly Rekognition finds a particular term - in this case, most of the files that were tagged as "outdoors" were, in fact, outside shots. There were a number of outside images that were missed.
See how auto tags affect search
Auto tags are factored into search results from the main search box. In most cases, this is how your members will be making use of auto tags. Type a term into the general search box and see what is found. You’ll notice that auto tags do not auto complete in the search box, unlike with regular visible tags. Members will just see a set of matching files (and probably some non-matching files as well).
Auto tags will be factored into search results, but the tags themselves are not visible to general members (unless you set the Site Settings to show to members).
Review the tags per image
When you double click on an image, you are taken to the Asset Detail View. One of the items you’ll see in the Quick Info is the set of auto tags assigned to the file. You can remove inappropriate ones if you like.
Review tags with lower confidence scores
You will also see a count of the tags that were below the confidence threshold. When you mouse over the count, you’ll get a popup showing all of these tags. As you will see, most of the tags below the 80% threshold are pretty unreliable. This helps you decide if auto tagging is useful, and if the 80% threshold is right for you.
You can see all auto tags that were assigned to an image in the Asset Detail View. Remove unwanted ones. You can also see all tags that were below the confidence threshold.
Converting Auto tags to intentional tags
The Auto tags can be useful as a tagging aid. You can filter on relevant tags and add them as “real” keywords to the appropriate files. This can make the tagging process go much faster.
Here’s how.
1. Filter on an auto tag that you would like to convert.
2. Review the shown photos, it’s likely that you don’t want to assign the tag to all files, due to mistakes in auto tagging.
3. Select the relevant files.
4. Make a matching "real" keyword.
5. You can simply drag the selected files onto the proper tag in the tag tree.
This GIF shows how to convert an auto tag into an intentional tag that you add to the taxonomy tree.
Evaluating NSFW tags
Library Managers, Content Managers and Admins will see the sensitive content tags, which we place in the NSFW (Not Safe For Work) panel. These tags can alert you to objectionable images, which you may want to delete or to keep hidden from general members. As with the auto tags above, the NSFW tagging is based on probability, and will not always be accurate.
Once you have run the auto tagging process, you can find all items with the sensitive content tag in the NSFW panel. Click on a tag to see the matching files to evaluate. Delete or hide any files that you wish to remove from general access.
Removing NSFW tags
You can easily remove the NSFW tags.
- Filter to the particular tag or to all NSFW tags
- Select the files you wish to remove the tag from
- Go to the Action menu, and select “No” in the NSFW tags dropdown
- Apply to the files.
- All NSFW tags will be removed and the image will be marked as Cleared
You can remove inappropriate NSFW tags by selecting files and removing the tags in the action menu. After removal, the files will be marked as cleared.
Evaluating Auto Transcription
The last AI tagging we’ll look at is auto transcription for video. This one is useful for almost everyone. When you upload a video file, Amazon Rekognition will automatically analyze the audio, and create a time-stamped transcription. This transcription is searchable in the general search box. You can also do a word proximity search in the filter panel, to find words that are near each other in the sound track.
To see the entire transcription, double-click into the Asset Detail View. You can turn on closed captions and follow along as the video plays. You will be able to check the general accuracy of the transcription.
When you open the Transcription panel, you will see a searchable version of the transcript. You can search for the word or phrase is question, and see all instances. Clicking on any line in the transcript will jump you to that point in the video.
You’ll see that the transcription has errors. The number of errors is dependent on the clarity of the speech, language and dialect, and the amount of slang or jargon in the video.
This GIF shows how auto transcription works with video files. You can search on a word (in this case, the word is "keyword") and see all files where this word appears in the soundtrack. You can also do a proximity search (in this case, where Keyword and List are within 100 words or 10 words of each other). You can double click on the resulting file and find where the term appears in the sourd track.
Wrap-up
We hope that the ability to see auto tagging in action will give you a better understanding of what is possible now, and how it might be useful to you. It’s also good to know what it can't do, so that you can set your expectations properly.
We have a number of other auto tagging functions planned, including Optical Character Recognition (OCR), face recognition, and use of different tagging services. We’re concentrating on the ones we think are most useful. In any case, we promise to always be straight with you, and to never over-hype the capabilities of the service.
We’d love to hear from you. Please feel free to drop us a line and let us know what you think, or what services you are looking for.
Cover photo by Amanda Dalbjörn