Auto Alt Text

Description

Use the power of AI to caption images with a simple right-click.
Let's make the web a more accessible place.

What is this?
Auto Alt Text is a chrome extension that can generate descriptive captions for pictures.

Currently, users who are visually impaired must rely on metadata and alt-text descriptions put in by website developers in order to understand what an image actually contains. However, not all web developers take the time to caption all their images. This is where Auto Alt Text steps in.

Using artificial intelligence, the extension can analyze an image and detect the contents of the scene depicted in it within 5 seconds!

How does it work?
It's pretty simple to get up and running!:

Download the extension
Right click on any image element (note does not work with background images at the moment)
Click "Get Image Info" from the dropdown
Wait a few seconds and get your caption

What is the tech behind it?
Auto Alt Text is based off of the im2txt model which was created by Vinyals et al for the 2015 MCOCO Image Captioning Challenge.

The model itself is based off of a encoder-decoder neural network (basically a deep conv net paired with a LSTM). The deep conv net first encodes an image into a vector representation using Inception v3 (a popular image recognition model). The LSTM then creates a captioning model based on the Inception v3 encodings.

I converted the model into an API and pared it down so that it could fit on a Lambda instance and stay loaded into memory for blazing fast responses under 5 seconds (compared to the > 15 seconds needed for the model to classify out of the box).