While browsing Facebook recently, I noticed a few friends sharing a selfie with a box below that offered a description of the photo. The text described everything in the photo, including everything in the background and the person themselves. While the background description was pretty mundane, the way it described the person themselves was pretty down to earth and not always flattering. In one instance, he said the person had a “menacing smile” and was “no eyebrows” (which is not the case). I did some digging and found the description was from a website called CLIP Interrogator. A website that uses artificial intelligence to generate a description of the photo you upload to it.
How the CLIP interrogator works
According to Noor Al-Sibai who writes for a blog called “The Byte” on futurism, the purpose of CLIP Interrogator is to provide a description of a photo if you are looking to recreate it. The best way I can think of to explain it, and I’m not a techie or a computer programmer by any stretch of the imagination, is this: Let’s say you find a photo you really like online, but that you can’t use it because it belongs to someone else, and that would constitute copyright infringement. So instead, you can upload the photo to CLIP Interrogator and use the description it spits out to stage your own version. Honestly, I don’t know why you couldn’t do that just by looking at the picture, but what do I know?
What happened instead was that people use it to see how the program describes their appearance. Considering it is an artificial intelligence and has no human emotions, it can be very brutal. Al-Sibai notes that he described a professor at Georgia Tech “like a muppet” who has a “punchy face”.
Of course, I would do it myself, and maybe even you since you took the time to read this article, a disservice if I didn’t try it myself, right? I took three selfies which I posted on my own Facebook page and uploaded them to CLIP Interrogator. The first was a photo taken as I sang Poison’s “Talk Dirty to Me” while wearing a blonde mullet wig and playing video game guitar at this year’s Celebrity Karaoke at the West Fall Festival. Side Nut Club.
Here’s how CLIP Interrogator described it:
In case it’s hard to read, here’s what it says:
a man holding a guitar and singing into a microphone, screenshot, inspired by Milton Menasco, featured on reddit, cobra, hair in a ponytail. shirt, blonde fur, pride parade, jurgen klopp, cascadia, corporate hip, long white hair, 80s, performing
There’s nothing here that hurts my feelings, although I have no idea who Milton Menasco is or what a cobra has to do with anything. My favorite part is “blonde fur,” which is misspelled to begin with, but is also pretty accurate for the rat’s nest I wore on my head that day. Let’s try another.
This one is from 2021 – 911 Gives Hope for the Holidays. I had made a bet with Travis Sams for the Colts vs Steelers game that if the Steelers won, I would wear Steelers gear in public. Obviously, my Colts let me down, so I paid off my debt.
Here is what he says:
a man standing in front of a truck in a parking lot, stock photo, by arlington nelson lindenmuth, pinterest contest winner, very angry expression, pittsburgh, blue – gray gear, iphone selfie, wafflehouse, mike mignola, sellers, early mood morning, trash, waporware
Not bad. I am (was) a man standing in front of a truck in a parking lot. I had a very angry expression because I was not happy to have to wear the equipment. And it was an iPhone selfie. Why it included “wafflehouse”, I have no idea.
Finally, I tried just a normal selfie. I took this one in the studio before the launch of our most recent St. Jude radiothon.
Here is what he says:
man standing in front of microphone, silk screen, inspired by Dave Arredondo, wearing modern yellow t-shirt, tumors, last selfie ever taken, radio equipment, ohio, dmd, symmetrical logo, back shot dive from behind
Again, I don’t know who Dave Arredondo is, I assume he’s a photographer and I accidentally took a photo that looks like something he would do. The line that concerns me is “last selfie ever taken”, which sounds rather disturbing. Does the AI know something that I don’t?
If you’re curious to see what it says in one of your photos, you can try it out for yourself here.
[Sources: Futurism / CLIP Interrogator]