Mailing List Sending FAQ

This FAQ specifically tries to answer questions about sending a whole lotta mail, to a whole lotta people, through Dada Mail.

What's, ``a whole lotta mail to a whole latta people?'' Depends on your server setup - if you're on a shared hosting setup, a lot of people could be 500. If you have a dedicated server, it could be over 25,000. Regardless, we're going to be talking about sending out messages to your entire list, instead of just simple how Dada Mail sends a single email message.

This FAQ goes through quite a bit of detail on how to use the batch settings and auto pickup features in Dada Mail.

Summary (What to do, without the, ``Why'')

Do this in the Manage List -> Sending Options screen:

Thankfuly out of the box, these are the default settings of Dada Mail.

One last step:

If you do all these steps, you will be on your way to trouble-free mailing list sending.


Sending Messages in Batches

What's ``Batch Sending''

Batch Sending is used primarily to slow down a mailing and space out message sending over a longer period of time. It has basically two parts to it:

During a mailing list message sending, Dada Mail will send a specific amount of messages out, wait a specific amount of time and then send another, ``batch'' of messages.

Batch sending does not work by sending a specific amount of messages during a specific amount of time. Close to that, but not that.

Batch setting is configured in the list's control panel: Log into your list and go to: Manage List - Sending Options. You should see the settings to change. Pretty simple.

If you'd like to set a default for all new lists, you may in the Config.pm variable, %LIST_SETUP_DEFAULTS. The key/values are:

Pretty straight-forward. For more information on the %LIST_SETUP_DEFAULTS Config.pm variable, see: http://mojo.skazat.com/support/documentation/Config.pm.html#_list_setup_defaults

Should I send my mailing in batches? What are the benefits?

Yes. Yes you should.

One benefit of batch sending is that you'll receive a message in your Dada Mail usage logs when a batch is completed, with some verbose information on the status of your list mailing.

It sort of looks like this (broken up into a few lines:

 [Fri Oct 13 00:44:41 2006]     listshortname   127.0.0.1       Batch Successfully Completed
 Subject:Davka Message, Start Time: Mailing Started: 10/13/2006 00:44:11
 process_has_stalled: 0 should_be_restarted: 0 first_access: 1160721851
 percent_done: 0 is_batch_locked: 1 last_access: 1160721881 id: 
 <20061013004408.77295734@example.com> total_sending_out_num: 1014 type: list   
 total_sent_out: 4

Handy.

Historical Note - Version 2.10.10 and below:

In older versions of Dada Mail, batches were numbered, so the logs would say, ``Batch #1 Completed'', etc. This is no longer the case, since mailings may stop mid-batch. As explained, this is OK behavior now. Mailings may also re-start mid-batch. This makes numbering batching slightly useless.

Why would I want to throttle down the speed of a mailing by using batching?

Good question! If you think about it, why wouldn't you want to send as many messages as fast as possible?

Sounds like a good idea, but consider these extremely important points:

Why are the batch settings so limited?

Currently, without hacking about, you can send 1-60 messages and then wait 1-60 seconds. Sounds like not a lot of wiggle room, but it should be plenty -

Consider that one message, every 60 seconds (the slowest setting you can have) is 60 messages an hour. That's slooooow.

Sending 60 messages, every second (the fastest setting you can have) is a rate of 3600 messages a minute - 216,000 messages an hour. If your mail server can even support that, hats off - unless it's specifically tuned to send copius amounts of mail, there's no way you'll be able to get up to that speed.

Honestly.

The fastest you're going to get, in pretty good conditions is 25,000 messages an hour(?) (untested, thus the question mark)

Hope I helped to dispell that myth.


Auto Pickup Options

The, Auto-Pickup Dropped List Message Mailings is going to be your best friend.

Basically, this feature, once enabled will make sure the process of sending out a mailing will succeed to completion.

You can configure the feature in the list control panel. Log into your list and under Manage List - Sending Options, you should see the settings to change.

Why do you need to use the Auto Pickup feature?

Sounds like this feature is something bolted on, to fix a larger, underlying bug in Dada Mail. Kinda right, but not really - here's the skinny:

Unlike traditional Mailing List Managers (Mailman, Majordomo, EZelm), Dada Mail does not run as a daemon - (and this is simplifying things) which is a program that is always running, just waiting to do something.

Daemons usually aren't allowed to run in shared server environments, so that's why Dada Mail isn't one. Shared server environments also usually have a set limit on the amount of memory a program process can consume, how large the process itself can be, how many files it can have open and most importantly for us, how long a process can run for.

So, that's the kind of environment we're working under. Quite restrictive.

This poses some problems, for this simple fact: Sending out a message to a lot of people takes a lot of time. If you've read the above, you've realized that you want to send your mailing list sendouts in batches. No question.

If your subscriber list is long enough to go over the limitation of how long a process can run for - which is pretty easy, your mailing is going to fail, mid way through somewhere, and this is bad.... if! you do not have a way to reload a list that has stopped at a specific part of your list, automatically. Which is what the auto-pickup feature does. So - no fretting.

Historical Note - Version 2.10.10 and below:

To give you some historical background, there was a time (gather 'round kids) when Dada Mail did not have the auto pickup feature, so the best you could do is try to figure out where your message dropped and manually restart it from a specific point in your list. Frustrating. Clunky. Dumb. (dumb programmer).

Times have changed and now we have the auto-pickup feature.

Should you use it? Yes. Please, yes. It's going to help you make a successful mailing list and it'll allow you to send a mailout slower, which, as we went over in the batch sending FAQ's is actually a good thing.

How do you use the Auto Pickup Options?

You can configure the feature in the list control panel. Log into your list and under Manage List - Sending Options, you should see the settings to change.

You have two settings to which to play with:

What is the Sending Monitor Screen and why is it so important for the Auto Pickup feature?

The Sending Monitor Screen will tell you how far along your mailout has gone.

In the list control panel, when you send out a message, by pressing the big green, Submit Mailing List Message button on the, Send a List Message screen (and friends), the screen will refresh and you'll see another screen, stating your mailing is on its way! and give you another big green button which is labeled, Monitor Your Mailing!.

Press that button.

The screen will refresh again to the Sending Monitor Screen

Keep the Sending Monitor Screen open until your mailing has finished to make sure your mailout goes to completion.

The screen should refresh every few seconds to update the status of your mailing. During a refresh, if Dada Mail sees a mailing has been dropped, it will automatically pick up the lost mailing at the exact spot it was dropped. And that's how the auto-pickup feature works.

If you are not on the Sending Monitor screen, the Auto-Pickup feature will not work.

If you are not on the Sending Monitor screen, the Auto-Pickup feature will not work.

If you are not on the Sending Monitor screen, the Auto-Pickup feature will not work.

During a mailout, is it safe to navigate away from the Sending Monitor screen?

Yes! Absolutely.

The Sending Monitor screen does not control a mailout - it only monitors and reports what's going on. If it finds a mailout has dropped, it will initiate the auto-pickup function and restart your mailing.

But! If a mailing is dropped, it won't be restarted, until you go back to this Sending Monitor Screen, so it is a good idea to keep it open.

But, if you do navigate away by mistake, it's not the end of the world, just navigate back.

How do you get back to the Sending Monitor screen if you navigate away?

Quite easily:

Log into your list via the list control panel. Under Send a List Message, click on, Monitor Your Mailings. This screen will give you a list of the mailouts currently active. Click on the subject of the mailout you'd like to monitor. And you're back.

Is there an alternative to auto-picking up mailouts, rather than the Sending Monitor screen?

Indeed there is, see the auto_pickup.pl.html extension. It is run by a cronjob. If this extension is installed and running correctly, you won't have to use the Sending Monitor screen at all.

I'm a nerd. Any documentation on the Mailout/Auto Pickup API?

Indeed, see: http://./Mail_MailOut.pm.html

in Dada Mail itself, see the auto_pickup.pl extension noted above, or the sending_monitor() subroutine in the mail.cgi file.


Reference: Various Hosting Companies and Email Sending Quotas

If you can help in fleshing this list out, it would be really helpful to me - contact us via the boards: http://mojo.skazat.com/support/boards or the mailing list: http://mojo.skazat.com/cgi-bin/dada/mail.cgi/list/dadadev/ and let us know!

Bluehost

http://bluehost.com

According to http://helpdesk.bluehost.com/kb/index.php

Problem Why do my email messages keep getting returned?

Solution If your emails keep getting returned with the message ``Unrouteable Domain'', then you have exceeded your hourly limit for outbound mail.

There is a default limit on all accounts of 50 messages sent per hour. This is a policy to help control spam on the internet and does not apply to your inbound email. You can receive as many emails as will fit in the accounts quota.

If you have a legitemate need for more email capacity on your account please let us know with a detailed explaination of why and we will gladly accomodate your needs.

Take note of that last response - it looks like they basically want you to let them know what you're doing. That's pretty responsible of them.

Also take note of this FAQ: http://helpdesk.bluehost.com/kb/index.php

Which states that you won't want to use SMTP sending on Bluehost.

Some notes from a BlueHost User:

Where does that leave you? Two options, I think:

Cyboretum

http://www.cyboretum.com

Currently, there is no limit to outbound email sending, within reason. Contact them directly if you have any q's.

Dreamhost

http://dreamhost.com

According to http://wiki.dreamhost.com/index.php/Smtp-quota

DreamHost enforces outbound mail quotas. While trying to send mail, if you receive an error message which mentions ``Quota Exceeded,'' you have surpassed your mail sending quota. If you hit this quota 2 or more times, your sending ability will be disabled until you contact support. In your support request, please include your email username and email address.

The outbound quota is 100 recipients / hour.

Please note this is NOT 100 emails per hour. A single email to 100 people in the CC/BCC fields would consume your entire quota.

For this host, you would definetely want to set batching, at a rate of 1 email, every 37 seconds or so. This limit is pretty strict.