I work for a company in a department that sends out on average, 2 million emails a day. These are emails to people who (1 or more):
- have signed up and are receiving a verification email.
- are giving us money
- are using our site at least once a week during the first month, and even if they taper off, are at least using it every 180 days.
So, we're not sending email to people who we've never met, as it were. We have an unsubscribe link in all our emails, and an account email settings page that has about half the mailings we can send as initially subscribed (we like money), but which can be turned off. We even have links to our settings page in the emails and on all the site pages.
I like to think we're being fairly responsible, all in all.
Like I said, we send 2 million emails a day. That goes out on a lot of machines, and we have lots of automation going on nearly every minute, and lots of email queues being prepopulated to send out in volumes acceptable to gmail, outlook, yahoo, etc.
So, you've asked to unsubscribe to the email you received last week. We got you, fam. I analyzed logs and did some live testing using our VPN at offices across the world. Over the past 3 years, you've been removed from your chosen lists within 2 seconds of clicking the button. This obviously doesn't cover hardware/network/server problems.
So we're cool, right? Nope. You're unsubscribed all right. However, we already placed you in the queue for our most recent mailing about 4 hours ago, and you're going to get the last one from us anywhere within 2 seconds from now to maybe late tomorrow.
You really won't get anymore from us after that, though.
It doesn't seem that complicated to remove a single entry from the e-mail distribution list of millions that's going out in one to two days. Harder problems have been solved. This doesn't adequately explain it.
The reality: it's financially a waste of time for a company to create an A+ off-boarding experience. Getting 1-2 extra emails isn't the end of the world: it's good enough for the company, you get off the list, they don't have to think about that unprofitable problem any further.
Most companies don't put much effort into off-boarding. Some go out of their way to make it even less palatable. But for removing yourself for a mailing list, I don't consider this to be the worse. Things surrounding payments are much more important: if I cancel service, you better not continue charging me.
Most people don't consider the full lifecycle of anything, and I wish they did. We've created a society, for example, that consumes disposable plastics, as an ordinary daily activity, and that ends up in our waterways as microplastics. It's easy to make things. It's an order of magnitude more difficult to manage the full lifecycle of the thing.
The technology behind it all could actually function perfectly but in reality an overworked marketing intern grabs a csv file from their desktop, opens it in Excel, compares and manually removes the unsubscribes (from another csv file that's delivered to them daily by email because that's easier) because "that's how it's always been done around here", then saves it as "June email push (1)(1) copy 2 (1)(1)(1)(1).xls" on their desktop, deletes all their subscriber lists in Mailchimp, as per the usual procedure, and imports "June email push (1)(1) copy 2 (1)(1)(1).xls" from their desktop, and sends out whatever.
So much this. These kinds of workflows are far more common than you would think they’d be, looking in from the outside.
This is true even if your company has in-house developers. At the small company I work for, we have barely enough developer bandwidth to cover most things related to our core products as it is. Improving internal support processes just isn’t even on the radar for us as a dev team. So those things either get contracted out (where we as a dev team may have limited or no evaluation input with regards to the quality of the solution), purchased (again with little or no input from us as devs) or are built ad-hoc by the less technical side of the business.
Companies have a lot of internal moving parts and resource limitations that together lead to these kinds of things.
Ah, i wrote my message out to my Kafka queue? Oh, there's not an easy way to remove it? Great, let's rearchitect our system around a different system that makes it easy to remove items from a queue once we've sent it out. Oh, there is none? Great, let's hire a bunch of experts who can write such a system from scratch that competes with the mainstream queueing systems out there. Or we could just say queued messages are queued, and tell our customers they'll stop receiving emails roughly a day from now.
If you (generally, not you specifically) architected a system that's brittle in the face of new requirements, that speaks to the resilience of the architecture to withstand change.
> Getting 1-2 extra emails isn't the end of the world: it's good enough for the company, you get off the list, they don't have to think about that unprofitable problem any further.
If I receive any further e-mails after unsubscribing, I mark them as spam. I don't think I'm the only one who does. That should be some incentive against sloppy unsubscribing mechanisms.
Because 1 in 3 email adresses is @gmail.com and google learns very fast from the Spam button. So if you want your emails to arrive to the subscribed people...
Not really my place to judge how your system is built, but double-checking an email is still subscribed just before you push it out from the queue isn’t impossible. At your volume, this will end up around 20 primary key requests per second, which is a decent database load, but then again for a company that needs to contact 2m customers per day it sounds like this wouldn’t break the bank.
Not letting people unsubscribe quickly is a conscious, commercial choice/tradeoff, not some kind of hard technical limit. Legislation can push this easily in a different direction.
At the point it's in the queue, it's already a composed email. Doing a file search and deleting the email file might be possible. I don't know how realistic that is though.
We built your customized email 4 hours ago. It's sitting in a directory somewhere waiting to be read and sent out to gmail. We're waiting because we already sent "X" emails to gmail addresses 3 minutes ago, and we need to wait "Z" more minutes. It might be in the list of emails to be sent as soon as the gates open up again. It might not be ready to go until another 50,000 emails before it have been sent.
So, let's say you've decided you don't want to get our emails anymore. Great. We've updated the appropriate tables. Next cron job will skip you. Now let's implement your criteria.
We've got to tell the MTA to figure out which is your email, and delete it from the queue. Hopefully we got to the MTA in time. We might not have. Remember the example I initially gave? You might be receiving the mailing 2 seconds after you unsubscribed. So we've spent some measurable time trying to prevent something that's already happened.
We don't have enough unsubscribers in general that this is going to cause a significant overhead. The thing is, I'm not even sure I can do that. Our routing application is a 3rd party commercial application that we don't have the source code to.
Not to mention, we're going to have to hop back and forth between machines as our databases are elsewhere. Not even sure they're in the same data center, now that I think about it. I really don't want to look up the NAT information to work this out for a simple internet discussion.
I'm gonna punt and say, "I dunno. I know how my part of the company works, and I'm still learning it. Maybe there's a better way to do it, and if you have information on commercial products that will do what you're stating is the desired goal, I'll do some research on them, and make recommendations to our company if they look viable."
I'm a C# developer. I'm still not quite sure how I ended up handling email. :D
If you were talking about the difference of a few minutes, then sure I'd let it pass. But hours? No, that's ridiculous. How long does it take to check if the destination email address of an outbound email has been unsubscribed? If it takes long enough to make it so you can't maximize your rate limit then you're doing it wrong.
> At the point it's in the queue, it's already a composed email.
So just...don't send it? I don't see why having done prior work to compose it means that you have to send it. You could argue that it's a waste of resources to have composed it and not sent it, but that's a sunk cost you can't get back, so why not just throw it away if you already know that's what I prefer?
> So just...don't send it? I don't see why having done prior work to compose it means that you have to send it.
So just heavily modify sendmail/qmail/whatever to maintain a list of email addresses for which it shouldn't send it?
Not associated with the grandparent, but the software likely composes and immediately "sends" the email, at which point it's going to be in MTA queue that's completely oblivious to the fact some email addresses have been unsubscribed since they entered the queue.
Overall, what you're suggesting sounds somewhat unreasonable. Once the email is "sent" (actually, added to MTA queue!), he might have no control whatsoever what the MTA does.
If you throw a stone, stopping it mid-air can be... hard. Perhaps it's doable. But is it a reasonable expectation?
I mean I agree but it's also possible to store a list of recently unsubscribed email addresses on the servers sending the mail out. Is it worth the trouble? Probably not.
You don't need to double check, you just have a way to remove emails from the queue. And doing simple removal from the queue only when someone unsubs is much less work than checking every e-mail in the queue against some central subscriber list, before sending.
The vast majority of bulk email is done via 3rd party providers like constant contact or mail chimp because delivery is important.
You can't just check an address--you have to check against the particular mailing list, email address and account which means you'd have to basically have to have the database. Otherwise, if I unsubscribe from "Megous Monthly Mailer" I'll stop getting the announcements from my kid's school with useful info like what the first day of school for my kid is.
So you need to know the account, the mailing list that was unsubscribed, the email address. Of course, the system is architected for high-volume, and scheduled email. Delivery volumes can be in the billions per minute. Anything too crazy and your service level is destroyed.
Queues are designed for high-volume queuing, not search, or retrieval. There's no "query API" for rabbitMQ, for example.
All of this is easy if we hand-wave away the requirements for high volume email delivery.
If n is the number of emails and k is the number of unsubs, it adds up to O(n) to check the table of unsubs before sending vs. O(k*n) to scan the queue after every unsub.
Or, to be less computer-sciency, a task that scans the queue of emails every time someone unsubs would be a pain to keep running. Elsewhere in this discussion the queue was described as a file. Who wants to scan the same file over and over all day?
Yes, but all that is easy to optimize. You can scan the mail file on entry to the queue and store file path/recipients to an indexed database. Then removal is just one SELECT query and unlink() call.
Nah it doesn't. Cache the list of emails that have unsubscribed, your dispatching process will filter out from that list.
Let's say 10million have unsubscribe, at 128bytes per email. That's roughly under 1.5 gigs of ram to store all those in memory.
Sending 2million emails daily comes out to an average of 23 a second. You can search 23 times through that list pretty quickly with one extra CPU core.
If it was sane company, you could. When I was doing some support for less sane companies, sometimes their "servers" were less powerful than my phone (hey, it still works after 10yrs, why change it). When my current company made product, everyone in such less sane companies was amazed that we could produce monthly route reports for fleet cars in seconds instead of minutes (and as devs we thought it was still not optimized enough).
I can see a case where the actual mailing is being handled by a delivery service and the ability to remove a single entry from a queued job might not be technically feasible because either the relay service is not exposing it to you or (and hopefully the actual reason) the delivery service only has the job that data stored in encrypted form and addresses are decrypted at time to send using a kms/kmip type of process.
I really don't think the possibility of receiving one more email rises to the level of needing new legislation and adding one more liability to help push people to moving all of the data and logic externally.
I don’t think it’s particularly onerous for a relay service to have an API for ‘remove all outgoing emails for this address’. It can even be hashed, because it doesn’t really matter what the address is, just that you know what mails belong to it.
I’m fairly certain that’s something postfix will do for you too.
Just because someone has unsubscribed from your mailing lists doesn't necessarily mean you want to stop sending them all mail. For example, we've got people who unsubscribed from our mailing lists but still submit support requests by email, using the same email that had been on our mailing lists, and they expect to continue to get answers to those requests.
In other words, We just don't care about users that unsubscribe (and why would you, really, you only care about profit) so we (unintentionally, maybe) engineered our systems so that unsubscribing is one of absolute lowest priorities.
In this scenario, I never willingly placed the order in the first place.
If Amazon started sending me a ton of useless random packages every week, you bet that I'd expect them to have a team of ninjas chase the driver down and forcibly remove my package.
Spam and marketing mail should be - needs to be - opt-in and illegal.
I would love to have an ACL on my physical mailbox, where all senders I have not explicitly authorized get mail returned to sender at their own cost.
If it takes up to 2 days for the system to send the first email then I think it's fair it takes up to 2 days for the system to send the last email. To some reasonable limit of course (i.e. just setting 1 year queue latency isn't a fair play).
Interesting problem. Of course I would still press the spam button upon receiving the second email and given that I use gmail, it would probably hurt your send rates.
Might want to fix that with a filter on the queues.
Well, at some point you took an architectural decision with an implication that if a customer requests to unsubsribe at time t, the request will be effective at t+N. This is a choice you made so the short of it is, it takes N becase it suits you.
Most of that question involves a level of detail I can't get into. But, we use the same batch of IP addresses for all our mailings. We have authentication set up with SPF, DKIM, and DMARC, and we're very careful on our rate of delivery, and as above, we respond quickly to unsubscribes. As well as the occasional report of our mailings getting labeled as spam.
We also partner with a reputation monitor that makes sure we're behaving, and they also let us know if we're approaching thresholds of not email friendly behavior. The only time I've seen that happen was when we did some IP consolidation and certain mail handling companies automated systems found our behavior anomalous, and thus was flagged suspicious. Since we knew, and our monitor knew this would happen, it was handled, and everyone (us, monitor, mail domains, customers) was running back to normal behavior within about two weeks.
In the end, no blacklisting from this, and none while I've been here. We've had a fairly high inbox delivery rate (based on our rep monitor, about 3% higher than the average) over the years, and about the only thorn in my side is Comcast, and even our reputation monitor has "special" metrics to measure their customers in regards to Comcast. :)
Ahh “open rates”. The great lie of the email marketing industry.
Those numbers that Marketo and company show are either completely made up or at least have hidden “scaling algorithms” applied.
All modern email clients block all image and CSS requests by default, and email security filters examine any linked URLs automatically. So there is no reliable “opened but did not click” feedback mechanism at all in modern email.
Set up a seed list of 100 email addresses you control, send through email marketing tool of choice. Open exactly 1 of those emails from a default client. See what lies their “open rate” dashboard gives you.
Ahh “open rates”. The great lie of the email marketing industry.
Those numbers that Marketo and company show are either completely made up or at least have hidden “scaling algorithms” applied.
All modern email clients block all image and CSS requests by default, and email security filters examine any linked URLs automatically. So there is no reliable “opened but did not click” feedback mechanism at all in modern email.
Set up a seed list of 100 email addresses you control, send through email marketing tool of choice. Open exactly 1 of those emails from a default client. See what lies their “open rate” dashboard gives you.
As was clear in context: Google evaluates the real thing for inboxing. It is relevant to which tab your emails get placed under. Hence all the advice to remove people who don't engage.
Whatever open rate stats are shown to marketers are irrelevant to google's deliverability choices.
Let's imagine you use 3 different marketing providers, plus an in-house one, because you're a big company and your teams all want to use whichever tool works best for them.
A user unsubscribes. This creates an entry in one specific system. In order for that to be reflected across all systems, it needs to be copied somewhere central, then synced back out.
Ideally you'd have a webhook hit a Lambda function and call it a day.
But, again, largish company with a gotta-move-fast employee growth mindset, engineering doesn't want to work on it (or, if not a tech company, you don't have in-house engineering).
So you hire some consultants who convince you that your email marketing is a "big data" problem, and they contract out the work on some Enterprise Infrastructure Platform as a Service product (an expensive, slow Lambda). The resulting system is slow, and often breaks, and you run it every few days in one bulk load/unload.
If they don't want to make it easy to unsubscribe me immediately I just report it as spam. Hopefully that goes into Gmail's whatever "big data" of spammers and starts getting them classified as spam across the entire network.
Can't speak for Gmail specifically but we ran a biggish email operation a few years ago (sending real estate alerts) and were notified when some of the big providers users marked our emails as spam.
So unless things have changed, the newsletter providers are notified.
Why not simply always report as spam? I have a zero strike policy with spammers. I’m not going to even try your “unsubscribe” link. You’re going to get marked as spam if you’re spamming me, and I’ll have the satisfaction of knowing you’re 0.000001% more along the way towards not being able to have your emails received.
Yea, and it’s insulting to even call it “unsubscribe” since I did not subscribe in the first place! The word itself subtly tries to shift blame onto the victim, as if it’s their fault they are getting spammed.
Like those Robocalls that leave you a voicemail featuring someone clearly reading from a script, intimating that they're "returning your call" about something you know for a fact you never called about?
Go away Janice, I don't need an extended vehicle warranty, and no I didn't contact you for information.
Not only do browsers often have zero days but many people configure their browsers to let the server do basically anything it wants. Service workers, WebGL, Webasm, javascript.
Any of those have been demonstrated to be capable of spectre/meltdown/etc. Hope your passwords/ssh keys aren't stored in RAM.
If I don't want it, and I certainly don't want any more, then it's spam. Doesn't matter if it's showing up because I forgot to unclick a checkbox somewhere and so isn't technically "unsolicited". Ain't no-one got time to help marketers lie to them. Try not sending me stuff I don't want.
It’s a bit extreme to say that any email that you don’t want is spam. I joined a mailing list years ago for an open source project I was interested in, and recently I stopped using it. I don’t think those emails suddenly became spam because my interest changed. Of course I unsubscribed through their system, I didn’t report it as spam.
I agree. And if I "forgot" to unclick a checkbox somewhere it's almost certainly because the website used some dark pattern to make it easy to get wrong.
> Let's imagine you use 3 different marketing providers, plus an in-house one, because you're a big company and your teams all want to use whichever tool works best for them.
> A user unsubscribes. This creates an entry in one specific system. In order for that to be reflected across all systems, it needs to be copied somewhere central, then synced back out.
The idea that it takes any longer to unsubscribe just demonstrates the marketing companies being user-hostile. Let me demonstrate why what you're saying sounds ridiculous:
A user is subscribed to a marketing provider (probably against their knowledge or desire). Minutes later all three marketing providers have the user's information.
If the system is slow, the marketing company would solve that pretty quick. If it often breaks, the marketing company would solve that pretty quick. If you run bulk loads/unloads every few days... maybe I could see that. Maybe. But the marketing company is likely going to want to fix that too.
How many users do this database have? Thousands? Millions? Databases from twenty years ago handled that just fine and quite fast too. Are you trying to tell me that modern databases are slower? Or are there perhaps too many users in the database from, eg, a dragnet?
Is the Enterprise Infrastructure Platform as a Service product truly to blame? Are you trying to tell me that the service is incapable of handling the ingress of a data point (user subscribed, user unsubscribed) in a timely manner? If that's truly the case then I have to truly wonder what else that service doesn't handle well: user privacy and information security are two top points that I'd wonder about. Competence and susceptibility to spearphishing would be some others.
Oh, right. Marketing companies aren't "user-hostile". Users aren't their target audience; users aren't their "users". Businesses are.
Company I worked at did have bulk loads/unloads every so often. This didn't need to be fixed for outgoing emails, or at least wasn't a priority. If you unsubscribed, you'd get taken on/off the generic newsletter-style mailings immediately. Things which were less common (e.g. surveys) could take a while. That wasn't an issue.
If someone who joined today didn't receive something from the survey mailer, that was a-okay. For unsubscribes, people didn't complain. I suspect that was because (1) Our customers trusted us. (2) Most customers received no more emails. The number of people who got a second email from some auxiliary system was very small.
But we couldn't guarantee immediate, full unsubscribes.
Regarding competency, security, and privacy, those are rare resources in the tech industry. Most companies couldn't care less about security of user data or privacy. Most also can't achieve competence.
One of the problems with unsubscribing is that I've seen a LOT of marketers re-using old lists and importing me back into a new list from some obscure snapshot, often with names like "new-list-feb-2019". There's no guarantee that, even when I unsubscribe from a given list, the company hasn't already exported my email address to some CSV file for future marketing efforts.
Or the linkedin approach of "oh, you opted out of featured posts and featured comments on featured posts but we've just invented a new category of "your three hop connections commented on a featured post" and you're included
Start marking unwanted emails as spam. These spam complaints would quickly put your email address into "Do not send" list of every bulk email sender who is capable to send deliverable emails.
Oh, and in case you're wondering how they did (similar) things in the Good Old Days(TM), let me tell you a story from the late 70's/early 80's.
I subscribed to "Cycle" magazine in my youth, and due to mistakes on their part, for a couple of years I got two copies, and when I finally decided to unsubscribe, I got only one copy a month (but for another year).
My mother had written to them to unsubscribe me (it was a recurring birthday present).
My first year of college I wound up meeting a guy in the magazine subscription business, and told him my story, and told me how it worked.
Turns out that they got a LOT of mail at the (US) publisher. So they had a machine with a gripper that grabbed each letter, held it while a grinder ground off (!) three edges of the letter, and then put it on a conveyer belt. One person was tasked with folding the top page of the envelope back so the letter was revealed, and a few people (IIRC) took them and put them in boxes, taping the letter back together (!) when part had been ground off with the edge of the envelope.
These boxes of letters were then sent to Ireland (!) to be processed, where people entered what should be done on some kind of mainframe application, which then cut 9-track tapes that were sent back to the US for processing.
Told this to my Mom, with the delightful result of seeing her collapse in tears of laughter.
Somebody tell that to Ticketmaster, I've unsubscribed from their shit at least half a dozen times over the last few months but I still get a marketing email at least once a week!
I've always assumed that the answer is that due to the way email works, mail can, in certain (rare) situations, end up stuck in a queue somewhere between mail servers and not delivered until a couple days later.
Saying that unsubscribing takes a few days means that in the off-chance that this happens, the sender has some coverage against annoyed users who have one of these mails delivered after unsubscribing.
I'm sure that's some of it, but it's also a result of lists being pulled ahead of time. Imagine you have 100,000 subs and 10,000 of them are going to get promotion A and then another 15,000 (with some overlap) promotion B. Often, the lists are pulled before the content is ready. Sometimes getting the final approval on marketing emails takes a bit, and so the person who unsubbed when they got email A are already on the list for email B.
It is very expensive to work with out-of-date email list.
If emails address bounced or, especially, if potential recipient is known for clicking "spam" button, then sending email to such recipient significantly lowers bulk email sender score.
I use a rule of thumb that every "spam" click costs me (as a bulk email sender) about $10. If somebody already unsubscribed -- it is quite likely that the next time they receive similar email -- they will click "Spam" button (which would cost me $10).
As someone who works in email I've never used a platform that's not using real-time lists or segments. A business could certainly do it this way but it would be a lot more work and a lot less effective.
Nice story of enterprise life. But personally, I still apply strict standards - if your unsubscribe link doesn't unsubscribe immediately, then you are spam, and will be marked and treated as such.
Pretty tough when it's your bank. I get marketing phone calls from Bell all the time trying to sell me on more TV channels. Joke's on them - I don't even own a TV! I tell them every time they call, they say thank you, and I get a call from them again every four weeks or so.
> - if your unsubscribe link doesn't unsubscribe immediately, then you are spam, and will be marked and treated as such.
I just never use unsubscribe links period. If I don't want something (and it's basically never anything I actually signed up for like a news letter or mailing list) I just have it dumped to the trash or a spam folder. I never see them anymore and it no longer impacts me and as an added bonus it consumes resources for the companies who spam my inbox. Why bother with an unsubscribe attempt at all?
It's because of permissions there's different databases, plus most importantly the drop lists are usually keyed up 48 hours in advance (you have to do in advance because of all the checks you have to do, etc.). They can take you out of the main but you're probably in some drop lists that have already been send to the provider -- that's why they say a few days/72 hours.
At least in India some companies sell/leak their user email list to others; those others also cultivate these details from domain names, company registration etc; & then fake or referred-link emails on genuine companies' name (with or without Genuine Company's consent).
I don't click on "unsubscribe". My standard procedure is to look up the sending IP-Address in WHOIS and send a note to the operator's abuse address. The good ones take reports about unsolicited mails seriously, and the bad ones end up on blacklists.
Based on the responses I (rarely) get, I've helped boot a few spammers from their servers by providing evidence to the operator.
At my company, we prepare one-off marketing and legal email blasts in advance, and need the final recipient list a couple days before sending. This allows time for processing the list for opt-outs, duplicates, etc.
That sounds like O(minutes) processing time with a reasonably written program/indexed db and a few million subs? And I’m sure you could get it to be faster than that. Based on all these comments, it frankly sounds like the real answer is that no one gives enough of a shit to do this right.
So often I assume a process is totally automated, but a lot of the time I should be more empathetic, because there is a person in the loop and they are usually just trying to keep the system they inherited from blowing up. They have no time to fix it.
This illustrates something that I think a lot of us in the "computer industry" often misunderstand. We see a mass email system (or anything happening at scale) and assume the whole thing is automated, because that's how we would do it.
Too often a system is cobbled together, only barely works, and is only semi-automated. Even something that is 99% automated but generates thousands of actions per day ends up creating a high workload for a human.
I've even seen where my email address was clearly hand-typed from a form (not even copy/paste), because I usually sign up for website accounts using Gmail's plus feature. I created an account at Website A with the address "myemail+websitea@gmail.com" and then received an email a day layer sent to "myemail+wesbitea@gmail.com" which Gmail still delivered because they ignore everything after the plus sign. The only way that error happens is if someone typed it by hand. [Plus emails are a great way to find out which companies sell your info to spammers. Most of the time nobody bothers to run a regex to fix "plus" addresses into the original address, so the evidence of data selling ends up right in the email header.] I feel sorry for the person that is eventually going to have RSI because they hand-type the entire list from their web form into their email software.
In our case we often have people write or call our global support number and request they be unsubscribed. They don’t have that ability be they create a ticket. That ticket gets sent along and eventually winds up in an admin’s inbox. That’s why it takes a few days.
This was a decade or so ago, but we used to get a list of ~2 million email addresses that we'd import into a new database and a single mailing would take anywhere from 2 to 3 days to complete. Sometimes it would take us a few days to get to it though.
The one to two weeks lead time always made a lot of sense to me.
(And yeah, you don't need to tell me how horrible everything about that process is... but it worked and no one's motivated to fix it. I wouldn't be surprised if that's the same process they're using today)
Creating a Mailchimp (or similar) account and using that for your newsletters doesn't take much effort and would be so much more efficient than the mess described.
Why are big and medium companies usually such a mess?
Is it because the bigger the company the less people care?
Maybe it's that the complexity and discipline required is simply too much for the average human?
Maybe companies do not have or are unwilling to invest the needed resources? (which ironically creates more waste)
Giving over your list to a third-party provider like Mailchimp can be a risk. One thing most publishers is more protective over than anything else is the database. Also, Mailchimp just isn't ideal for high volume mailings/companies who do this as a revenue generator. It's designed more for the mom-and-pop operations.
Ok, Mailchimp was a bad example (we actually use Sendgrid for our newsletters) but my point was about using a service that solves this for you instead of having such a convoluted process.
The reason it takes a few days is because the next mailshot you're going to be getting in the next couple of days will be sent out by some third party to list of email addresses which was exported to a csv file and sent unencrypted to their gmail account yesterday.
I used to work at an email service provider that managed email marketing campaigns for some pretty large companies. It's been quite a number of years now, but I don't imagine things have changed all that much.
Mostly, it's just CYA language because of the way the various old and slow systems work, plus the CAN-SPAM act legally allows up to 10 days to process an unsub.
There are multiple checkpoints that prune lists as they get churned through the machine, so typically you'll be fully unsubbed within 24 hours (often much less), but they don't catch all cases at all times.
The abbreviated process for sending out a marketing campaign at a large ESP typically looks like:
- Marketing manager makes a request for a list of people that match x/y/z analytics criteria (e.g., purchased within last x months, typically opens email, geographic region, etc). Depending on how "sophisticated" the criteria are and how backed up the analyst department is, this may take a couple of days to get turned around.
- The list of addresses is created and then pruned down by known unsubscribes or other do-not-email constraints in the system at the time the query runs.
- The resulting list gets sent out for review and approval by the marketing manager and client (how many people are we going to mail, what is it going to cost, what sort of metrics do we expect, etc). Since this is a human-in-the-loop process, it may again take up to several days to turnaround.
- After approval, the list gets churned through the unsubscribe list again, dropping any new unsubs. This step should catch new unsubs within that "it may take up to few days" window mentioned in the title.
- The final list is then queued up for sending, which depending on the size and meter rate may go out over the course of several hours. If you've already been queued up your unsubscribe request is typically going to get missed for this run.
Now, add on the complexity of syncing up multiple databases between the ESP and the customer, which is typically a nightly batch job at best. So even though your unsubscribe hit some web server instantly, it may take a couple of days for it to fully filter through from the web server to the client's marketing databases into the ESP's database. It's similar to why banking and ACH is so terrible: it's just ancient design patterns and slow process and nobody wants to pay money to modernize it. And if they miss a few unsubs they are still well within the legal bounds so it's whatever.
tl;dr: A lot of email marketing still runs on chains of batch jobs which can introduce windows of unsynchronized lists getting sent out.
It 100 percent is still this way. But generally your queued list can be set up 1-2 days beforehand, so that's why it's often 48-72 hours, because you're already in that list that's been cleaned at the provider for drop.
- have signed up and are receiving a verification email.
- are giving us money
- are using our site at least once a week during the first month, and even if they taper off, are at least using it every 180 days.
So, we're not sending email to people who we've never met, as it were. We have an unsubscribe link in all our emails, and an account email settings page that has about half the mailings we can send as initially subscribed (we like money), but which can be turned off. We even have links to our settings page in the emails and on all the site pages.
I like to think we're being fairly responsible, all in all.
Like I said, we send 2 million emails a day. That goes out on a lot of machines, and we have lots of automation going on nearly every minute, and lots of email queues being prepopulated to send out in volumes acceptable to gmail, outlook, yahoo, etc.
So, you've asked to unsubscribe to the email you received last week. We got you, fam. I analyzed logs and did some live testing using our VPN at offices across the world. Over the past 3 years, you've been removed from your chosen lists within 2 seconds of clicking the button. This obviously doesn't cover hardware/network/server problems.
So we're cool, right? Nope. You're unsubscribed all right. However, we already placed you in the queue for our most recent mailing about 4 hours ago, and you're going to get the last one from us anywhere within 2 seconds from now to maybe late tomorrow.
You really won't get anymore from us after that, though.