Number of Messages in a POP3 Inbox

There is often confusion about the number of messages in a POP3 inbox (for a given user account). Here are the typical scenarios that cause confusion:

  1. If your email client such as Outlook or Mozilla Thunderbird seem to show more messages than what Chilkat can see, then make sure you realize which messages are stored locally by Outlook, and which messages are actually on the POP3 server.  The intention of the POP3 protocol is that messages will be downloaded and removed from the server and managed locally.  (The intention of the IMAP protocol is the opposite: Messages are generally kept on the server and managed there.)  This basic difference in semantics is the reason why the POP3 protocol is so much simpler (i.e. bare-bones functionality) as opposed to the much more complex IMAP protocol.
  2. If viewing emails in a web-based email client such as GMail, also be careful about which emails are *actually* in the POP3 inbox vs. what emails have been removed from the inbox and are managed by the webapp (i.e. GMail) on it’s server.  (Similar to Outlook and it’s local store.)
  3. Many mail servers provide both POP3 and IMAP interfaces.  The POP3 interface only has access to the  INBOX.  A POP3 client will not be able to see the other  IMAP mailboxes.
  4. Your POP3 session has access to only those emails that existed at the start of the POP3 session.  If more emails arrive during the session, a POP3 client must end the session (typically disconnect) and re-start a new session to see the new emails.

How to Verify Exactly how many Emails are in the POP3 Inbox

A POP3 server responds to the “STAT” command with a one-line response indicating the number of messages in the INBOX, and the  total byte count of all messages combined.   To see this information, start a POP3 session by calling the mailman.Pop3BeginSession.  Then call mailman.GetMailboxCount to return the number of messages.  It will return the exact number of messages that existed in the POP3 mailbox at the time the session was started.  You can examine the mailman.Pop3SessionLog property to see the STAT command and the raw response.  For example:

**** Connected to mail.chilkatsoft.com:110
< +OK Welcome to MailEnable POP3 Server <576.1827327234@xxxxxxxxxxxx>
> USER ****
< +OK
> PASS ****
< +OK
> STAT
< +OK 54 2260671

Here we see that the INBOX contains 54 messages, and the sum size in bytes of all the messages is 2260671.

In addition, a POP3 server responds to the UIDL command with the full list of UIDL’s (one per line) for all the emails in the INBOX.  You may call mailman.GetUidls to get the full set of UIDL’s. The number of UIDL’s should match the number returned by GetMailboxCount.

If you examine the Pop3SessionLog after calling GetUidls, you’ll see something like this:

> UIDL
< +OK
< 1 2608A7A521EA454FB4B5FE790982286B
2 680602D898AF4946B73570C9C4B3512B
3 6982607A73C24948AC9781254ABC8797
...
52 F85CBF78337443D6BD7EF16D9B1609BD
53 3AB8CBEFF6B648A08C274F312FF1AB77
54 B06EFE3B1890462BAD391301F9109278
.

GMail POP3 Authentication — Must Unlock Captcha

If captcha is enabled on your GMail account (and it is enabled by default), then GMail POP3 authentication will fail with this error (even though a correct login/password is provided)

**** Connected to pop.gmail.com:995
< +OK Gpop ready for requests from 57.215.212.123 c23pf1097242ibu.22
> USER myaccount@something.com
< +OK send PASS
> PASS ****
< -ERR [AUTH] Username and password not accepted.

To successfully authenticate with pop.gmail.com from an email client such as Outlook, Thunderbird, or an application using Chilkat one must first unlock captcha:

  1. From the computer on which your signins are failing, visit http://www.google.com/accounts/DisplayUnlockCaptcha
  2. Enter your Gmail username and password, and the letters in the distorted picture.
  3. Once you have successfully signed in, retry running your application using Chilkat.

Note: Unlocking captcha is not required to access a GMail account via the IMAP protocol (imap.gmail.com).

POP3 over HTTP Proxy

ASP: POP3 using HTTP Proxy
SQL Server: POP3 using HTTP Proxy
C#: POP3 using HTTP Proxy
C++: POP3 using HTTP Proxy
MFC: POP3 using HTTP Proxy
C: POP3 using HTTP Proxy
Delphi: POP3 using HTTP Proxy
Visual FoxPro: POP3 using HTTP Proxy
Java: POP3 using HTTP Proxy
Perl: POP3 using HTTP Proxy
PHP: POP3 using HTTP Proxy
Python: POP3 using HTTP Proxy
Ruby: POP3 using HTTP Proxy
VB.NET: POP3 using HTTP Proxy
Visual Basic: POP3 using HTTP Proxy
VBScript: POP3 using HTTP Proxy

Understanding GMail’s Behavior

This blog post will explain GMail in a way that will help you understand it’s seemingly strange behavior. The question that prompted this post is as follows:

I am able to read mails from GMail mailbox using MailMan object. Is there any options to delete a mail permanently from GMail mailbox? When I am using the DeleteMail method the mail is going to the Trash folder, however I want it to be deleted permanently.

To explain GMail, I’ll first describe Outlook and then compare it to GMail.  Even though you already “know” Outlook, please read this because it’s important when comparing it to GMail.

Outlook is an email client application that runs on your computer.  It communicates with an email server to download emails.  The emails you’ve downloaded may or may not be deleted from the server, depending on your Outlook settings.  Outlook keeps it’s own “database” of emails saved locally on your filesystem.  These are the local folders: Inbox, Trash, Junk, etc. and the folders you’ve created.  Emails may be deleted from the mail server (POP3 or IMAP) but they will still be present in Outlook’s local storage.

GMail has the exact same model.  (I’ll explain)   GMail is an email application that runs on a Google server (not on your local computer).  The user interface is within a web browser.  Just like Outlook, the GMail application (running on Google’s web server) is an email client that communicates with a mail server.  In this case, it is Google’s mail server offering both POP3 and IMAP access.  (pop.gmail.com and imap.gmail.com)

Just like Outlook, the GMail application communicates with the mail server to download email messages, which it stores in it’s own local “database”.  (I double-quote the word database because it’s really just some sort of proprietary persistant storage of emails.)   What you see in the GMail web interface are the emails NOT on the mail server, but in the local folders of the GMail application — just like Outlook.

Looking at it this way should clear up the confusion.  If you delete an email via the GMail web user-interface, you are not deleting the email on the mail server,  you’re deleting the GMail application’s local copy.  If a POP3 or IMAP client (such as Chilkat) deletes an email from pop.gmail.com or imap.gmail.com, it is NOT deleting the GMail application’s local copy.  Expecting a POP3 or IMAP client component/library to delete the GMail application’s local copy is the same as expecting the component to delete Outlook’s local copy.  Email clients communicate with email servers.  Outlook and GMail are email client applications.  The Chilkat email component is also an email client.  Clients do not talk to clients.  Clients talk to servers.  GMail is a client, it can talk to pop.gmail.com and imap.gmail.com.  The same goes for Outlook, Mozilla Thunderbird, and the Chilkat components.

If you look at the GMail POP Download settings, it should now make sense to you:

GMail POP settings

The GMail application offers options just like Outlook — but the terminology they’ve chosen is confusing:

“Keep Gmail’s copy in the Inbox” really means “Do not delete the mail server’s copy”.

“Delete Gmail’s copy” really means “Delete the email from the mail server”.

I’m not sure what “archive GMail’s copy” is — I’d need to experiment.  It probably means something like: move the email to another folder on the mail server.  Perhaps an archive folder which can be accessed via the IMAP protocol.

Exchange Server – POP3, IMAP, and SMTP

Question:

May I ask if is possible use Chilkat components to connect to MS Exchange 6.5 and higher ?
I am thinking about purchase of MailMan component or other as it is, however I would like read some example in C# for reading emails from Exchange and sending emails via Exchange
after connecting to the specified account domain/logname & password.
May I see some examples somewhere on your site?
May you recommend correct components please ?

Answer:

Exchange Server (including v6.5 and higher) implements the server-side of the POP3, IMAP, and SMTP protocols.  Therefore, any software that implements the client-side of these protocols will work with Exchange Server in exactly the same way as any other email server. (See the note below regarding NTLM authentication.)

The purpose of a protocol is to allow any client-side implementation to communicate with any server-side implementation.  As long as each side correctly implements the protocol to specification, everything works.  For example, a web browser implements the client-side of the HTTP protocol.  You may not realize it, but it communicates with many different web server implementations: IIS, Apache, etc.  You can browse the web because the both clients (i.e. web browsers) and servers correctly implement the HTTP protocol.

Therefore, all of the examples involving POP3, SMTP, and IMAP at http://www.example-code.com are examples that will work with Exchange Server.

The Chilkat Email license ($99 for a 1-Developer license) is what you would need for the POP3 and SMTP functionality.  The Chilkat IMAP license ($199 for a 1-Developer license) includes all three: POP3, SMTP, and IMAP.  Note: 1-Developer does not mean “1-Computer”.  It simply means that one person may develop programs using the component.  All Chilkat licenses allow for royalty-free redistribution, meaning you may use the component/library on any number of computers.

*** MS Exchange implements an additional authentication mechanism called “NTLM”, which is also known as Windows Integrated Authentication.   Chilkat supports NTLM for POP3, IMAP, and SMTP.

Delete POP3 Email Based on Date / Age?

Question:

Is it possible to delete email sitting on the POP3 server based on its age?
For example, have it only delete mail from server older than 5 days.

Answer:

The POP3 protocol itself does not have the direct ability to delete email based on date/time (or any other selection criteria).  The POP3 protocol only provides the ability to delete email based on a message ID.  Therefore, your application would need to download emails (or download headers-only), iterate over the emails, and for each email where the EmailDate (or LocalDate) property is old enough, call the MailMan.DeleteEmail method.

POP3 or IMAP for MS Exchange Server?

Question:

I would like to use your product to process mail on an MS Exchange server. Which product should I use; Chilkat MAIL or Chilkat IMAP?

I saw in your examples that I can use both to connect to an Exchange server MAIL with Pop3 and IMAP for a direct login on the server but which one is the best choice? I have an application that should process all the mail that is delivered to an Exchange mailbox, save it to the file system and delete the message in the mailbox. The other task of the application is to create a new message, let the user interact with the message content (add or delete text) and send the message. Preferably with the user interface of Outlook.

Answer:

You may choose either — both POP3 and IMAP are valid possible choices.  There are some factors that may guide you one way or the other:

  1. Your Exchange Server administrator may have disabled POP3 or IMAP, so  you may be forced to choose a specific protocol if one of the protocols is disabled on the server.
  2. If users are accessing the same mailbox in Outlook, then make sure they are not downloading and deleting the email from the server.  This would prevent your program from reading the email on the server if Outlook already removed the email from the server.
  3. With POP3, the only way to create a new email on the server is to send email (via SMTP) to the mailbox.  With IMAP, you may create a new email and append it to the mailbox via the Chilkat.Imap.AppendMail or AppendMime methods.
  4. The IMAP protocol does not allow for an existing email stored on the server to be modified.  To “modify” an email, you’ll need to fetch it, modify it, upload the modified email (which will now be a new email with a new UID), and then delete the old.

Not all POP3 emails seen?

Question:

Our problem that in the POP3 incoming account there are 400 mails but the developer’s UIDL query can see only the first 251 mails and downloads only 251 mails.

Answer:

There may be several common reasons.  The first thing to do is to verify number of emails in the mailbox by calling the mailman.GetMailboxCount method.  It sends a STAT command to the POP3 server.  The POP3 server responds with a single line providing two pieces of information:  the number of emails in the mailbox, and the sum of the sizes of all the emails in the mailbox.  If the POP3 server replies to the STAT command saying that there are 251 emails, then there are only 251 emails in the mailbox on the POP3 server.   You may view the raw POP3 response by checking the Pop3SessionLog property.  For example:

**** Connected to mail.chilkatsoft.com:110
< +OK AVG POP3 Proxy Server <584.540060890@IPDMCZ0250MIA> 8.5.268/8.5.278 [270.11.4/1978]
> USER sales@chilkatsoft.com
< +OK
> PASS ****
< +OK
> STAT
< +OK 62 322571

More reasons for not seeing the expected number of emails:

  1. You’re using GMail.  By default, once an email is read by a POP3 client, GMail hides the email from view for all subsequent POP3 accesses, but it is still visible via the web interface.  There is a GMail account setting that can be modified to prevent this behavior.  See this information:  GMail POP3 Access
  2. You’re viewing the POP3 mailbox using Outlook or another email client and some of the emails you see are not actually on the POP3 server, but have already been downloaded and removed from the POP3 server.  You’re seeing the emails already saved to your local computer.
  3. When a POP3 session is established, you get a snapshot of the mailbox that does not change.  If new emails arrive, they will not be seen until the session is ended and re-started.  To do this, you may call Pop3EndSession followed by Pop3BeginSession.
  4. You set the MailMan’s MaxCount property to prevent downloading more than a maximum number of emails.

GMail Settings for POP3 and IMAP Access

The most common problem with reading email from GMail using Chilkat’s POP3 and IMAP API’s is the following: The user can see certain emails when interactively logged onto GMail with a web browser, but cannot see those emails using Chilkat’s POP3 or IMAP component.

The problem is caused by a setting in your GMail account. The default setting causes an email to NOT be seen once it has been downloaded by a POP3 or IMAP client. You’ll need to change your GMail account setting for “Forwarding and POP/IMAP” access. The option to choose is this:

When messages are accessed with POP: Keep GMail’s copy in the Inbox

See the screenshot below:
GMail settings

No X-UIDL header found

The following mailman methods use the X-UIDL email header as the means to identify the email to be downloaded or deleted from a POP3 server:

GetFullEmail
DeleteEmail
DeleteBundle

If no X-UIDL header is found within the email, then it is not possible to determine the sequence number of the email to be downloaded or deleted. (POP3 commands to fetch or delete an email require a sequence number. The sequence number is unique only to the POP3 session. The POP3 UIDL command is used to get a list of UIDLs and their corresponding sequence numbers — allowing a UIDL to be mapped to a sequence number. The methods listed above use the X-UIDL header field value to find the sequence number to be deleted.)