Friday, August 28, 2009

Kill Your Signup Form with Rails

Nobody likes Signup Forms

Even though the gradual engagement meme has been around for a while, and everyone just hates signup forms, they just seem to keep popping up like a bad habit. My site, was one of the guilty parties. We saw users coming back to the site repeatedly, but not signing up. The percentage that looked at the signup form and then bolted was uncomfortably high. It was time to kill the signup form. This blog post documents how we implemented gradual engagement using Ruby on Rails and restful authentication.

Gradual Engagement Principles

Here are some principles I've gleaned from learning about gradual engagement.

  • Make sure the user sees a direct benefit to every sign up step. When you do ask for information, do so only after 1) the site has demonstrated user value, 2) the user has a vested interest in the site, demonstrated by having customized some aspect of its behavior, and 3) you provide a carrot that justifies providing signup information.

  • Set reasonable defaults automatically and let the user customize them later. For example, in our original join workflow, Newsforwhatyoudo asked users to add subscriptions and topics as part of the process of joining a group. Instead we now automatically give users the most popular subscriptions and let them change these later.

  • Security only when its needed. Forcing a user to establish a username and password up front may be necessary if you're developing a banking site where all the information being displayed to a user is confidential, or the user's actions have financial impact. Most web sites don't have this issue. Even E-commerce sites can let users browse, search, compare and price without any security - its not until the credit card information is required to make a purchase that formal signup is required. For most sites anti-spam is a bigger issue, and there too, anti-spam measures like captchas and email confirmations should be used only after the user does something that requires their use. I emphasize the word "after", because if you wait until after a user has filled out a comment form, then present the confirmation, you're more likely to have the user perform the confirmation because they've committed energy to filling out the comment.

Keeping out the spammers

The most time consuming part of implementing gradual engagement was figuring out how to keep the spam bots out. At our strategy is two tiered. Actions that change state - particularly those that submit forms - require the client to run javascript. This eliminates most spam bots. If we detect a client that's not running javascript, we throw up a captcha. Most users never see the captcha and can just submit the form. The captcha eliminates bots that can run javascript. None of this helps if someone designs a bot specifically to attack our site, but then again neither did our old process of asking for user credentials and showing a captcha, so we've gained usability without losing anything.

Here's an example with a "email support" form.

   <% form_for :ask_support, :url => {:controller => 'home', :action => 'email_support'} do |f| -%>

><%= label_tag 'Your Email *' %><br />
= f.text_field :email %></p>
<p><%= label_tag "Email message *" %><br />
= f.text_area :message %></p>

<%= render :partial => 'shared/check_for_bots' %>

><button type="submit">send</button></p>

<% end -%>

The check_for_bots partial is added to any form, link, or button that changes state. This partial looks like this:

   <% if @show_captcha && ENV['RAILS_ENV'] != 'test' %>
<p style="padding-top:10px"
<%= label_tag "Help us reduce spam by filling in the below" -%>
= recaptcha_tags %><br /><!-- @human =true by default if were showing captcha -->
<% elsif ENV[
'RAILS_ENV'] != 'test' %>
<input type="hidden" name="turing_a" id="turing_a" value="Please do not alter" />
<%= javascript_include_tag
'turing_a' %>
<% end %>

check_for_bots adds a hidden input field to post and put requests. The "turing_a" javascript file runs when the form loads and returns a value in the hidden input field based on how long the user viewed the form before hitting submit. If the length of time is below a threshold or incorrect, we reject the input and instead show a captcha. After that the standard captcha acceptance routine applies. The javascript we use is adapted from Stephen Hill's Javascript Captcha.

// stop bots that cant run js or are too quick to submit the form
// check hidden value for a reasonable interval: interval in seconds

var turingA = function() {
if (document.getElementById("turing_a")) {
a = document.getElementById("turing_a");
if (isNaN(a.value) == true) {
a.value = 0;
} else {
a.value = parseInt(a.value) + 1;
setTimeout("turingA()", 1000);


Back on the server side we need to validate the hidden input field. If validation fails, we render a captcha.

def email_support
@popular_groups = Group.most_popular
raise MissingFields unless params[:ask_support]
@ask_support = params[:ask_support][:email], params[:ask_support][:message]
raise MightBeBot if !@human
raise MissingFields if || @ask_support.message.blank?
"about page",
params[:ask_support][:message] )
flash[:success] = "Got your request, we'll be in touch"
rescue MissingFields
flash[:error] = "Couldn't process your request, make sure both fields are filled in."
render :action => 'about'
rescue MightBeBot
@show_captcha = true
flash[:error] = "Your browser must be running javascript, and/or you must enter the correct words below"
render :action => 'about'

The check_for_bots method is added to any method we need to protect. It sets @human which the caller can check as part of its input validation.

  # Used as before filter on POST and PUT actions to determine
# whether the requestor is likely to be human. Requestor needs to be able to execute .js
# and wait at least 4 seconds before submitting the form, or this fails and returns false.
# This method doesn't require the user to do anything, other than have a js capable browser
# Note when testing in rails this method will (and should) always fail.
def check_for_bots
if current_user && current_user.status == 'verified' # already showed captcha and user passed
@human = true
# verify_recaptcha is true if in test mode or success, false if fails.
# verify_recaptcha may only be called once per action and get a valid result!
if params[:recaptcha_challenge_field]
@human = verify_recaptcha
if current_user
self.current_user.status = 'verified' # for other actions set it here.
@verified = true # for NewsC#more there is no current user yet, so tell auto_create_user
# to set status
elsif params[:turing_a].to_i >= 3 # 3 second delay
@human = true
@human = false

Auto creating users

The next part of getting rid of the signup form is to automatically create user objects, sessions, and remember-me cookies. This is what our old signup process did. We encapsulate this process into a method and have it run whenever an un-authenticated user tries to perform an action that requires a user object/session. We are using restful-authentication, and pulling out the relevant lines of code from there and from sessions controller yields this handy method:

  # Create user object if doesn't already exist.  Set cookie.   
def auto_create_user!
user = User.auto_create_user_object # doesn't save to db as handle remember cookie does.
user.status = 'verified' if @verified # user passed captcha, prevents showing captcha again
User.current_user = self.current_user = user # !! now logged in (sets session)
handle_remember_cookie! true # sets cookie and saves user so they can get back after session is over
# cookie set to 5 years.., the above uses @current_user set in previous line.

Note that since the user doesn't have the ability to "log in", you'll want to set your remember-me cookie time span to an appropriately long time. This can be changed in vendor/plugins/restful-authentication/lib/authentication/by_cookie_token.rb Once the user object is auto-created, the user can return to their accounts page and add a login and password if they want to use their account on another computer. We auto-generate a random username for display purposes, letting the user customize that later if they want to.

That's it! The old signup form still works, but is no longer necessary to use all the site's functionality. We've been in production for about two weeks now with zero spam infiltration, but time will tell. If anyone has better ways to do any of this stuff, give some comment love below.




James said...

That was a really great how to, and a thought provoking topic. Minimal buy-in before functionality is offered is a great way to get users hooked.

Damien Le Berrigaud said...

Really great article. :)

the Scan Artist said...

KK I'm putting it on my calendar to mail this to the Rails Rumble organizers next year. Before the competition. Last year they tried OpenID, this year it was a free-for-all again. It always feels weird to go through 20 registration forms for test applications -- knowing you'll probably never use it again. Perfect application for gradual reg.

csbartus said...

it might be offtopic, but your site looks scarry ... i had to run away after a few seconds.

a more friendly design could help also the signup process ....

Logan Henriquez said...

@csbartus There are several usability issues that we're starting to tackle - mostly around simplifying the number and presentation of controls and view. Anything particular you found "scarry"?

Anonymous said...

Hi Logan,

great post (I learned something about signup forms! such an important theme), great site and great concept, I particularly liked the top-bar that shows up when you visit an external link.

Here are a few comments regarding what cbartus might have meant:

-your join and signup buttons are not very attractive.. color/shape. people like you and I are not designers. We need designer friends.

-your black on blue title header is wrong...

-generally speaking, there is a lot of font inconsistency between your header links, footer links and sidebar links.

-in your main section I personally would flip the order: drop down boxes first, explanation after.

-finally my last comment would be that any 'social' site -and yours is to a certain level- that has an essentially static homepage is flawed: the homepage must be live, even if it means showing irrelevant posts, you must be showing, somewhere, in a corner if you like, the latest posts, or most popular or ...

that's my 2 cents, thanks again :-)


Anonymous said...

Hello from Russia!
Can I quote a post "No teme" in your blog with the link to you?