Panning for gold in the sewer: Fun with internet critique
Friday’s post about Scott Kurtz’s take on interpreting critical feedback eventually generated a couple very interesting comments.
One thing everyone agreed upon was that that the “signal to noise” ratio for feedback (incorporating all pro/amateur/literary critique/”you suck” e-mail) for any creative work on the Internet is astoundingly low. So the question remains – as a creator of any subjective work on the Internet (comics, films, poetry, bonsai kittens) – how can one filter the responses one gets to get useful information out of the mire?
I remember a film I was involved with premiering at the Toronto International Film Festival once and our target was to sell out the festival screenings (I think that would probably be around 800 seats over both shows) and try and hustle enough press to get a half-dozen writeups across the country. Filtering feedback from (at most) 806 individuals is trivial by comparison.
Contrast that with Basement Accident. (RocketAce has been doing a series of short viral comedy pieces with the imponderables both for their (truly spectacular) live shows, and to generate some new RAMP content while we’re developing other stuff). “Basement Accident” was a *really* quick and dirty affair (I think it maybe took a couple nights) just to see how the Internet would respond to a mash-up parody of two of it’s most beloved tropes (horrific accident videos and bad lightsaber effects). Within hours of posting it racked up almost 750,000 views on the various sites we post to (youtube, funny or die, college humor, break… et al) – and over 1,000 comments ranging from “gay” to “supergay” to:
Our nations future: filmmakers. And they couldn’t even turn off the time code display on the camera. Jesus, this is why TV and Movies are sh*t today. Don’t quit your day job!
You get the gist. I could pan through every single one of those comments and (seemingly) not come up with a single useful nugget. Or even a complete sentence.
Shortly thereafter blogs and forums started to chime in, only to complain about our lightsaber physics and damage model. Suffice it to say, there was not a lot of powerful media analysis going on there. If only someone in the mainstream media had brought their more considerable resources to bear on looking at the construction of… oh wait, nevermind.
Anyway, I chalked the whole experiment up as a resounding success (conclusion: the type of people who like “fell off the roof” videos feel personally cheated when they think they’ve been rick-rolled) and went on with my day. Some time later when I got a phone call I realized some of my creative collaborators were getting really bummed about the feedback. “Everyone hates it”, “Some guy thought I looked like a lesbian”, “they didn’t find it funny at all”… and I realized that I hadn’t provided anyone else with the filters to interpret the stream of effluence they were receiving.
With that as background, here are some simple tips for interpreting (or ignoring) seemingly useless on-line feedback:
1. Look at the big picture
If you held a big gallery showing of your work in a public square, and out of every 100 people who came to see it, only 1 had something nasty to say… you’d be over the moon. But on-line there is rarely the impetus for people who liked something to spout off. More often than not the people who feel the need to respond are those who just like to bitch about stuff. They are fairly easily identified, and easily ignored. If you don’t have sufficient comfort in this kind of distinction, go find a comments thread for a piece of work you think is amazing and then look at the number of negative and nasty comments it’s received and note the similarities in tone and (lack of) insight. In the case of basement accident, the views:negative feedback ratio was upwards of 500:1 but looking at fifteen pages of “you suck dicknipples” doesn’t really help convey that.
2. Decide in advance what type of feedback you will listen to (or not) and for how long.
One of the (sad) points from Scott’s “why we insulate” blog post was Mike Krahulik becoming discouraged about a new coloring technique because of negative feedback he had gotten (this especially rankles as my personal favourite Penny-Arcade strips tend to be the ones where he pushes artistic boundaries the most). If you’re going to try something new, decide beforehand what type of feedback you’re initially going to listen to, and what you aren’t. Or at least for what period of time.
It almost goes without saying that audiences don’t like change initially. If you’re going to change a style, approach, composition, or tone, go ahead and shut down what input you listen to off the start. If, months later, you’re still getting feedback that something isn’t working for someone – maybe that then has some more credence than the day you make the change. My goal for “accident” was to try and rile up a certain sub-set of viewer. Which is exactly what happened. So results I considered a “big win” were seen by others as a disaster. The type of feedback I considered importance was volume and vitriol… not necessarily what they were *saying*.
3. Don’t lump all feedback together.
One of the nice things about the Imponderables videos is that we often get to see them performed at live shows before putting them on the ‘net. This is great as it often gives us a chance to make a tweak or two depending on how the crowd reacts, or comments we may get from the audience after the show. However – a live audience is not an on-line audience. They both interact with material in different ways, and often respond very differently to the same sketch. If you’re a cartoonist your web audience is not your print audience is not your blogger audience, is not your collected trade audience (although there is overlap, obviously). Heck – even different audiences in the same situation responded differently to identical material! “Accident”, for example, plays very… strangely… for a live audience. I’ve seen it twice and one time it’s killed, and one time it was dreadful. Partially it assumes that a viewer be somewhat familiar with internet video and partly it’s short and dark… not the type of thing you expect to see in a theatre or club venue. The point is, the amount of information I could get for that particular pieces primary audience in that particular venue was limited.
4. Adjust for Bias.
By the same token, the things which are important to one’s peers, collaborators, bloggers, mainstream reviewers, your drunken audience, your sober audience, your wife, your mother, and your arch-nemesis are all quite different. Peer creator audiences tend to focus more on technical proficiency that is often unnoticed by a casual audience. Family is far more forgiving than strangers. Your arch-nemesis hates everything you do and will destroy you if it’s the last thing they do. Each avenue is potentially a valuable source of information, but each avenue has a unique and specific biases. If you count on your audience, peers, family, magic 8-ball for critical feedback, know (or make an effort to determine) where their biases lay and adjust for that.
If you apply some variant of these points to all avenues of critique, criticism, and comment you receive – then a critical review, blog post, even a random “you suck” e-mail, can be as valuable a source of information to a creator as they wish – if they are willing to filter it to fit their needs.
Where I’ve never disagreed with Scott’s post is that there’s no necessity for anyone to include *any* channel (any channel outside of the artist themselves) in their creative process – provided that they’re getting critical input somewhere. No one avenue of feedback has any more, or less, validity than any other, but part of learning to create in any field is finding what tools work the best for each individual creator. Tools to interpret feedback are just as diverse, important (and often, messy) as the physical markers, paints, clay, cameras, and computers themselves.
[edit 08/11/08 – Re-wrote several sentences to clarify and fix some really ugly grammar]
