Troubleshooting Troubleshooting
Who's fault is it?
The first time you get errors on your safelist hosting network, you are apt to think the problem is with the script. In some cases, you may be right. But in the majority of errors that occur with safelists and safelist hosting, the problem is with your hosting environment. It may also be to do with poor script configuration.
If you get error messages on certain pages, they should help to pinpoint whether there is a fault with the script, with the server (such as "too many connections", "unknown function"), or with your configuration ("invalid login", "username mismatch", "permission denied").
Whatever the error and the cause, if you would like us to look at it for you, please send as much detail as possible. Include a copy and paste of the actual error message, as well as the login information for your site in order to see the error reproduced. For example, if this is an error occurring in your member's login, please provide us with a member login in order to check it out.
We will also need your FTP/cPanel details in order to fix any configuration problems.
If you are on a dedicated server, we will need your root login in order to fix any server issues.
Please submit any support tickets through our new support desk at Ezy Help.
Please don't automatically assume that you have a "bad install" or a "bad script". Read our pages on the right hosting for your safelist hosting network and make sure your server or hosting account supports those requirements.
Thanks for your understanding.
Typical Problems
Why do I always have a high server load?
Before we start discussing the causes of high server loads, I'd like you to take a minute and have a think about the computer you are using at home right now. Most likely it's a fairly decent computer with a fair bit of disk space. Maybe it's a Pentium 4 or even one of the newer Dual Core PC's. Home computers typically have either 512Mb or 1Gb of memory (RAM). I now have some questions about your home computer:
- Do you think your computer runs poorly?
- Does it slow down when you try and open a few different programs at once?
- Does it sometimes "crash" when you run certain programs?
- Do you ever have Internet connection problems?
- Have you ever been infected with a virus (even though you use Anti-Virus software)?
If you answered "No" to all the above questions, congratulations it sounds like you have a well-running computer. I'd now like to add some things to your computer (in theory) and see how well you think it will keep up.
- I'm going to add another 25 people all using your computer at the same time.
- I'm going to get your computer to run 20 different programs at once.
- I'm going to have your computer sending 500 emails a minute in the background.
How well do you think your PC is coping now?
Ok, the point of all this is - your SERVER is a PC. Just like the one you are sitting at now. In some cases, your server is a little higher performance (speed and memory) than your home PC, but the majority of the time, your home computer is BETTER than your server. Now we're going to look at what you are asking your server to do and why it sometimes is unable to cope.
What is Server Load?
When computers were first introduced, they could only do one thing or task at a time. For a long time now though, computers have been able to "multi-task". This means we can have it doing several completely different tasks at once and the computer manages to 'pigeon-hole' them all separately and handle each task independently. For each task we ask of it, the computer may require a number of different processes. Take sending an email for example - it requires processes that connect to the Internet, load your email program, locate the receiver's email server, send the email, check for viruses and so on.
Naturally, if we ask the computer to do too many things at once, it might suffer from performance issues. The first of these performance issues is causing the LOAD placed upon the server to go UP. In technical terms, the load refers to the number of processes (or tasks) currently running or waiting to be run.
A high server load is not automatically a problem. In some cases, this load will be dropped almost instantaneously as the processes are completed. But, if your server has a task that has become "stuck" or has many tasks that are consuming a lot of CPU time, then the server load will simply grow and grow as more and more tasks are added to the queue of those waiting to be run. It is therefore good practice to prevent adding more tasks to the queue until it has dealt with the tasks already running.
When the computer is in this state, you will quite often see messages such as "Too much load" when sending or receiving email, "Server busy" when submitting safelist messages, "too many connections" when accessing databases. This means you need to wait until the server has caught up with what it's doing before it can take any more requests.
Why is my server load always high?
From the above discussion, you'll realize that the more things we ask your server to do at once, the higher it's load will be. On a safelist network, the most common task we ask of the server is to receive and send email. Notice I said "receive" first. Before your server can send out an email, it must receive it first. When you submit an email message, it is "received" by the server. It then follows whatever instructions it needs to in order to send it to the desired recipient.
If we submit a safelist message for sending it is quite possible that the server will receive 500 emails for sending (one for every member of the list). That means we are asking it do 500 things at once. Asking it to do too many things at once will cause the server to become sluggish and if pushed too far - it may cause other important services to stop functioning.
The server load is calculated every 60 seconds, so on a decent, well-configured server the sending of 500 emails would take less than a minute and so doesn't affect the server load at all. Now imagine increasing that to sending 50,000 emails in one minute. That will affect the server load of ANY computer. You don't think you would ever send that many emails at once? Let's consider the safelist submitter...
A submitter user is a member of all your safelists (all 100 of them on your server). They log in to their submitter and click "Send". Your server receives the request to send 500 emails (one for each member) to each list all at once. That's 50,000 emails right there. Increase the number of members, number of lists, number of submitters, number of safelist networks... and you can see the number of emails increasing dramatically and quickly.
Do you send solos? If at the same time the submitter user is sending email, you start your solo send, you have probably added another 60,000 emails to the task list.
A high-performance, well-configured server will process around 40,000 emails in one hour. Remember that sending an email doesn't just require 1 process. It requires several processes for each email (looking up recipients, checking for viruses, connecting to mail servers, etc.) I think you're beginning to get the idea of just how much we are expecting our safelist hosting servers to cope with.
Now let's say that you have a popular, busy safelist network. Your server would be receiving emails from submitters, members, admins and yourself fairly regularly throughout the day and even into the night as automatic cron jobs are run and other time zone users access the lists. It is not uncommon for a network of 100 safelists to be receiving (and attempting to send) over 1.5 million emails a day. Think about that number! 1,500,000.
So to answer the question "Why is my server load always high", it's basically because you are asking your server to do too many things at once.
How can I reduce the server load?
The only way to reduce server load is to restrict the number of processes (tasks) that the server is completing at one time or give it enough time between tasks to "catch up" and finish the taks already running. In the case of safelist networks, it means to limit the number of emails being received at once.
The most important action you can take to reduce the server load is to actively manage the blocking of safelist submitters regularly. Safelist submitters are the natural enemy of the safelist host! As we showed before, you only need one submitter member submitting to all lists to have 50,000 emails submitted simultaneously. Some submitters don't use "real-time" sending and instead send all their queued emails by cron job later in the day. While this can be great for the submitter's server, it can be deadly for you. Imagine, instead of sending 1 user's submission to your 100 lists (x 500 members), the submitter is now sending 10 user's submissions to your 100 lists all at once ... half a million emails!
You MUST continually be on the lookout for new submitters that may be submitting to your lists. You may notice them in emails received from your safelists, or you may come across them in advertising. Check whether or not your lists are in their submitter and block them! Don't haggle with them for payment. Just block them. Yes, the money may be good from 6 different submitters sending to your lists, but the costs in server maintenance, upgrades and downtime is not worth it. Limit your approved submitters to 2 well-respected ones only (3 at the most).
Click here for an updated list of new and/or moved submitters
The other thing you can do to reduce server load is to take care when sending solos to all your lists. If you regularly send 4 or 5 solos at once, make sure you pause at least 15 minutes between each send - even more if you have a lot of lists or members on your server. If you have more than 100 lists with more than 500 members each (on average), split them up into "groups" and send to group 1, pause 15 minutes, then send to group 2 and so on.
You should also consider restricting your admins to allowing a maximum of 5 solo ads per day. I have seen admins sending 30+ solos a day to their lists. Apart from being unethical, it causes a HUGE strain on your server (30 emails x 100 lists x 500 members=1.5 million emails).
Why don't I receive emails from the lists?
There are literally dozens of reasons why you are not receiving safelist email, but I'll assume that you WERE successfully receiving emails and now you're not. If this is the case, the most common cause is because the server is not able to send all of the emails it receives.
Just like the server load gets "backed up" by tasks waiting to run, the email queue can get backed up by emails waiting to be sent. As I mentioned above, your typical well-confnigured safelist hosting server can process (send) around 40,000 emails per hour. If you're keeping up, that's almost a million emails a day. Not all emails the server receives can be sent immediately. Of course, there's bouncing emails, servers not responding and so on. These emails are "held" in the queue where the server can attempt to send them again later.
Remember also that having a high server load may mean that your server may not process emails immediately (because it's too busy), so these are also held in the queue. It's very easy to end up with a mail queue of over 300,000 emails waiting to be sent when the server never has time to catch up or is in a constantly "high" state.
The emails in the queue are processed at regular intervals. The default is to process the queue once every hour which is insufficient for safelist hosting. But even if the mail queue is processed every 5 minutes, it will NOT process the queue if the server load is too high. Therefore, the emails end up in the queue and.. never get out. If your server is constantly getting in to this state, you really need to reduce the amount of emails (submitters, solos, lists, etc.), make sure your server is well-configured, and consider upgrading to a more powerful server or adding more memory to cope with more processes.
Why am I running out of disk space?
Ok, so you have a huge 160Gb hard disk drive on your server which appears to only be 20%full, yet you get error messages such as "Not enough space left on device". Why? Usually because of the email queue.
Depending on your hosting provider, dedicated servers are usually configured to have separate areas (partitions) for storing different types of data. There is usually one for your data (called "home"), one for temporary files, one for logs and statistics and so on. The emails in your email queue need to be saved for sending later and they are usually stored in the partition where logs are held. Usually this is only a fairly small partition (say, around 6Gb). If you have ended up like the server in the previous discussion and your email queue is over 300,000, then you will eat up all the available space in that partition and get error messages. This is an urgent situation. If there is no space left, then no more mail can be sent. Get your server admin to see to this immediately.
The other reason for using up all of the disk space is due to the log sizes (in the same partition). By default, a lot of your server's log files are not automatically cleared or rotated and your server admin will need to incorporate regular rotation of logs or they will become huge and eat up all the available space. On a safelist hosting server, it is important that your mail logs and queues are cleared or rotated DAILY and not weekly.
What does 'exim failed' mean and is it important?
Each time your server processes an email request, it starts an exim process. Each time it attempts to deliver mail in the queue, it starts an exim process. Depending what else is going on with your server (such as processes being killed off due to high loads), the "exim failed" message usually only means that one of the exim processes failed. It doesn't mean that mail is not working at all.
Is it important? Not really. If you have a busy server you can expect a lot of these messages throughout the day. But if you start to see a huge increase in the number of these, you might want to get your server admin to see if there's any possible cause for the additional failures.
My server was running well and now it's not. What happened?
One of two reasons:
More mail...
It's possible that a new batch of submitters have cropped up, or your lists have been added to established submitters. As we have discovered, submitters can result in massive amounts of emails. Check your server/domain logs or read your safelist emails to see where the emails are being submitted from.
The other cause of a lot more mail suddenly appearing on your server is if you have new admins now sending dozens of solos. Check the admin emails being sent by staying subscribed to your safelists.
Server Configuration...
If you have a well-configured server, then your server admin has spent considerable time modifying configuration files for web services, email processing and database handling. If performance suddenly deteriorates, it is usually due to one of these configurations being reset back to the default settings (which are NOT suitable for safelists). As an example, cPanel servers are set to be automatically updated with new releases as they are made available. This can result in your current configuration being reset. Unfortunately some cPanel updates can NOT be prevented no matter how hard your server admin tries.
In this case, contact your server admin to check on your configuration and re-optimize the server if necessary. It usually only involves adding back the files that were over-written and shouldn't be a large task (unless you have waited for a long time and a big cleanup is also required).
How else can I help keep my server running smoothly?
Remember the maths. Think about how many emails your server is receiving and sending and try to limit whatever you can.
For cPanel users, log into your WHM and view the Email Statistics (NOT the email queue).This will give you an idea of how many emails were received by the server over the last couple of hours, and how many it was able to deliver. If there's a large discrepancy (i.e. 50,000 received and only 30,000 sent) then your email queue is growing at a rapid rate.
On the Statistics page you can also check the number of bounces. If it is more than 10%,you need to do something to resolve it - delete members with bad emails, ban blocked domains, etc.
Why can't I check the email queue?
If you have an email queue of over 100,000 and you use your WHM to "Manage Email Queue",it will attempt to load all those emails. It actually READS them all from the disk and displays them to you and this on it's own can cause the server load to increase dramatically. You may want to take a peek just to see how many are in the queue - don't! WHM keeps loading the email queue in the background anyway, even if you stop loading the page. If you look again in a couple of minutes, the same thing happens. So you can end up with your server trying to load the email queue 10 times all at once causing your server to become even more overloaded.
Ok, so you really want to see how many emails are in the queue?
I'm not sure if I should tell you this or not. Not because I don't trust you. Well, yes,it's because I don't trust you.
First, you need to have root access or superuser permissions to log into your server via SSH. If you don't have that or have no idea what I'm talking about - stop right there!
If you can login to your server as root, type the following on the command line:
/usr/sbin/exim -bpc
That will give you the total amount of emails in your mail queue. Now type 'exit' and log out. You should not use your root login for the server except in emergencies.
Summary
Hopefully this page has given you some insight into what happens on your server and what you can do to minimize disruption and improve it's performance. If you have any comments or questions on this page, please submit them to our Help Desk at Ezy Help.
