We’ve been discussing ways to grow our email lists.

A little “bell” in the back of my mind jingled and reminded me about pop-ups as a mechanism to increase the speed at which we accumulate email signups. The “pop-ups” I am thinking about are those boxes that appear over what you are reading and make you an “offer you can’t refuse” in exchange for your email address.

But pop-ups are evil, right? No hospital marketer would ever use one. The only people who use those are sleazy marketers looking to make a quick buck.

Not only would they damage our brand they don’t even work. I would never enter my email address into a form after seeing a pop-up. However, being one for data-driven decisions, I could not help but do some research before deciding. This is a summary of what I found.

1. Do pop-ups make readers flee?

I thought pop-ups are plain evil but the more I looked into this the more I came to think that it’s not that cut and dried. A better characterization is that pop-ups can easily be misused, and have been so frequently, that we think of them as annoying.

The facts suggest that if you use pop-ups carefully and they really offer something of value then they may not be perceived as bad at all. Dan Zarella, the author of “The Science of Marketing” tested email sign up pop-ups on his blog and found that the bounce rate did not vary when using pop-ups. This data suggests that readers were not fleeing in horror at the sight of his pop-ups.

2.    Are pop-ups actually effective?

A big counterargument on pop-ups is that they seem to be effective in growing emails lists in a large percentage of cases. So if you’re an “80/20 type marketer” you really will want to take a look at pop-ups.

As you can see from Dan Zarella’s data above the his bounce rate did not change when using email pop-ups but his sign up rate changed a lot, in fact it more than doubled.

Here’s some more data on the effectiveness of email pop-ups (from Suggarae, a popular digital marketer and blogger). The inflection point in the graph below occurred when she added an email pop-up to her site.

3.    Know your audience

It turns out the lift pop-ups give you vs. the annoyance they cause seems to be highly sensitive to the setting and your audience. Copyblogger does not use pop-ups, reporting that their audience sees a lot of them and has grown “allergic” whereas they have seen other audiences don’t mind them at all. It seems this all comes back to testing in your own unique environment.

4.    Pop-up for “scrollers”

The pop-up you probably want to avoid is the one that obscures content the moment your visitor hits your site. These are too intrusive for most scenarios. A better implementation is likely a pop-up that triggers only when a user has scrolled through a certain amount of content or has spent a certain amount of time on your site.

Finding the exact amount of an article the reader needs to have read, or the amount of time on your site, will take some testing. Here’s a graph on the timing of pop-ups from Crazy Egg.

5.    Pop-up for “exiters”

Another good implementation of pop-ups is for people that are exiting your site. Giving people a chance to join your email list when they are about to exit seems to accelerate email acquisition significantly and there is not much to lose as the people are leaving anyway.

6.    Warning small screen

If you do implement an email sign up pop-up you need to be careful about its implementation on mobile devices. It’s easy to fill the user’s entire smartphone screen with a pop-up without intending to do so. Many systems allow pop-ups to be suppressed on mobile devices.

7.    Not if they did it

Another common implementation problem with pop-ups is that marketers don’t turn them off for people that are already members of their email list. This is frustrating for readers as they are asked to sign up for an email list they already belong to. So if people have already done what you want, make sure you don’t show them the pop-up.

8.    Test and test again

Everything I found came with the warning that each website and each audience are unique. You won’t know whether pop-ups will work for you, or how to implement them, until you test and test again.

The conclusion of my research: Pop-ups are not inherently evil, but the devil is in the details of how you implement them.