2.11's main feature will be an overhaul of the mass emailing routines. The problem with the current sending routines is that there is no meta information saved about the sending process, thus, if/when a problem does occur during the process, there's no way to automatically pick up the mailout. This is further made delicate, because in Dada Mail, it's a one shot thing - a mailout starts, and if it doesn't finish, it doesn't finish. This is a really bad spot to be.
This needs to be addressed before the multiple fields project (Ver 3.0, basically)
Some more details:
Problems:
So, I propose the following changes, which are all quite dramatic:
Currently, a mailing list message is sent by first making a temporary list of all the subscribers and some extra meta information to fill in some of the tags - to personalize the message, in other words. This is great and we're going to keep this file around.
What's going to change is the addition to a few new files:
Knowing where you are on the sending will allow you to stop, pause and restart a mailing from a specific, known point.
If we have a copy of the message being sent around, we can solve a whole lot of problems.
These reasons all lead to difficulty in actually debugging a bad mailing list message mailing (say that 5 times fast)
So, that's all a headache. To put this all in motion, here's how a mailing list message will be sent to the subscribers:
You create the message, hit, ``Sent to my subscribers''. Dada Mail says, ``OK, great''.
Dada Mail makes a tmp directory for this specific message. The temp directory holds:
Dada Mail will read in the tmp copy of the message, and send messages, one by one. Each message could be sent with a function that wraps in an eval block, so that if there's a problem its caught.
No problem - counter gets +1 added. Problem? Counter still gets added, problem logged.
And this goes on, until sending is complete - the counter file = the total # of subscribers. Once this happens, Dada Mail cleans up after itself.
What happens if there's a problem? Many of the problems with sending happen are silent, which makes it very difficult to debug.
Problems can be found by the new logging mechanism being applied (above) and by seeing if the message is finished sending and if the process that was created to send the message is still up and running.
If there's problems, the program could automatically jumpstart sending - starting with the last email sent, using the tmp copy of the message and just keep chugging along. nothing to it. This could happen until the message is complete.
There would also be enough information to see where in sending the message is, give an estimate on the time needed to finish, and an option to kill the sending process, without having to kill the actual sending process using the, kill command.
Dada Mail, version 3.0 will be basically Dada Mail with multiple subscriber fields support. This seems like only one small feature, but it affects every part of the subscription process, it affects how mass mailings are created, how you administrate the list itself, most everything, except, perhaps archiving. Big. Deal. Very big deal.
Also, if you want to look at the 4+ history of the Dada Mail 2.x serious, you'll notice that the program is entirely different than it was when released as 2.0 - it's basically has been rewritten a few times, in chunks. So it goes.
For more information on multiple subscriber fields and Dada Mail, see:
http://mojo.skazat.com/features/multiple_fields.html
More information will be released as the feature gets fleshed out.
Or, things we really really want to work on, but haven't had time/money (commission us!!!) to get to work on.
At the moment, Dada Mail's release cycles have been tied to the linear fashion that it has been developed:
----> 2.8.12 ----> 2.8.13 ----> 2.8.14 ----> 2.8.15 ----> 2.9----> 2.9.1 ----> 2.9.2
This causes major problems: for example: any bug fixes in 2.8.15 couldn't be released until everything needed in 2.9 was completed. The time between 2.8.15's release (November 17th, 2004) and 2.9 release (May 11th, 2005) was almost four months! That's too long for simple bug fixes to be released. Currently, (August 8th, 2005) we're again going long in the tooth with the release schedule.
This impacts many things, including people's interest in Dada Mail:
etc,
What should happen is a scenario like this:
There should be a stable release and then any new work that is feature-related will be for the next major release - any bugs found in the last major release will be fixed in its own branch and released as x.x.1, x.x.2, etc. Bug fixes will then be merged into the next major release release. For example:
branch!->/2.10.1 ---- 2.10.x\ / \<-merge! 2.9 ----> 2.9.1 ----> 2.9.2 ----> 2.10/----------------------\2.11
This will allow us to release bug fix version, that have no new features added, which will lead to fewer surprises from people that don't like surprises in their software, like ISP's and other people that deploy Lots of software.
This idea isn't novel at any length of the imagination. If I'm talking greek, you may want to brush up on CVS branching:
http://cvsbook.red-bean.com/cvsbook.html#Branches
Questions: Why are we using CVS? Because that's what Sourceforge offers. I'm aware of alternatives - some that work much better than CVS, but none are offered by Sourceforge ATM. This may change.
Subject: This is the Subject! And here's the message!
The list control panel should have a way to edit these that's much better than what's there right now - each message should have its own screen, instead of what's there now: ALL the messages in ONE screen.
clean and customization-friendly
There should be two options when moderation is enabled:
[] Moderate all messages [] Moderate messages not from list subscribers
And then, moderators should be, ``called to duty''
[] Send a notice of approval to ALL moderators [] Send a notice of approval to a random moderator
The, ``Notice of approval'' is just an email saying, ``hey, this message needs approval''. The message then has two links, one to approval, one to deny and a copy of the message.
I'm also tempted to have an option similar to, ``Send a notice of approval to a random moderator'' but:
Send a notice of approval to a random Subscriber
So, a list could moderate itself. Sounds like a more community-focused thingy.
This would only work if there was some sort of Spam-Prevention hook in Dada Mail. I'll talk about that in my next point, but if this feature was enabled, you'd also want to send a message back to the poster stating, ``hey, you're not subscribed, so you're not going to receive any more messages from this list, why don't you subscribe? And the necessary link to do just that.
http://search.cpan.org/~felicity/Mail-SpamAssassin-3.0.4/lib/Mail/SpamAssassin.pm
It's three lines of code to get the status of a message. That's a great API.
If the message is SPAM, you could either:
If this was in place, you could have an open list and not feel like it would be suicide.
Only have this option available if the message is less than x days old (like, 7)
Have anyone who enters a valid email be able to reply to a message, but send a confirmation link to the email itself to verify that it exists. The message could also (or only!) give an option to subscribe to the list as well.
This would really make Dada a better communication tool and sort of have one step between the archives being a sort of forum - another mailing list that's also a message board, that's also a blog, that's also a (sort of) CMS. Ohh.
Some disadvantages I see:
I guess it wouldn't be too too hard to accept messages straight from STDIN, as mojo_send.pl/dada_send.pl did, although there goes some of the security/infinite loop checks that have been really nice in dada_bridge.pl
Another option would be to accept messages from a http POST - this would facilitate communication with the dada_bridge.pl script and outside programs... I guess.
The only hurtles that have to be jumped over to make this work is:
This was different than in dada_send.pl, where the list email had to be:
We are NOT going back to that.
It would be interesting to add more features that are taken from weblogs, like:
http://example.com/cgi-bin/dada/mail.cgi/archive/listsn/2005
Would show all messages from 2005, or:
http://example.com/cgi-bin/dada/mail.cgi/archive/listsn/2005/04
Would show all messages from April of 2005
All these navigational schemes do not take into mind threading, which is sort of a discussion list/advanced message board (like Slash) idea. That's fine with me - it's just another way of grouping information.
The other option is to somehow create a new paradign that's either some sort of hybrid of sorting by date/sorting by thread or something so totally completely different that'll blow your socks off.
If you start out with a score of, ``10'', it would take 10 soft bounces to get unsubscribed from a list.
A hard bounce, because the mailbox just isn't there, may give you a score of, ``5'', or ``10''.
There's nothing intrinsically hard about this, but there is the issue of where the bounce scores are to be saved. Currently, you cannot save them in the subscriber DB, because it only supports the email address.
You could create a new db mj-listname-bounces (say), and just base it all on DADA::App::GenericDBFile, which is exactly what that module should be used for. We could add a new method for ++'ing a value of a key and be done with it.
After the 2.10 release, I'd like to talk more about moving some or most of the documentation into a Wiki that can be edited by anyone. This will stop old lazy bones here to hold up the availability of documentation to the project and give us another thing to play with.
I probably need someone to step forward as an editor.
Anything else that should be done?