EmailEngine supports various email backends to interact with users' email accounts. This document outlines the different protocols and services that EmailEngine can connect to, including their specific requirements and considerations.
Email backends that EmailEngine can connect to
EmailEngine supports various email backends to interact with users' email accounts. This document outlines the different protocols and services that EmailEngine can connect to, including their specific requirements and considerations.
IMAP and SMTP are the most common protocols used to access email accounts. Nearly every email hosting provider supports these protocols. EmailEngine uses IMAP and SMTP as the default connection methods.
Many OAuth2-capable email hosting providers support OAuth2 authentication over IMAP and SMTP. This means that when requesting permissions to access an email account, a standard OAuth2 authorization flow is used. When a user grants EmailEngine permission to access their email account, they are granting access to IMAP and SMTP using OAuth2 tokens rather than traditional username and password authentication.
When EmailEngine establishes an IMAP or SMTP session, it uses the OAUTHBEARER or XOAUTH2 SASL extensions to authenticate, similar to connecting to a regular IMAP or SMTP account.
For Gmail, the required OAuth2 scope is https://mail.google.com/. Note that this is a "restricted" scope and requires a mandatory security review. Unless your use case requires the ability to permanently delete emails, it is unlikely that Google will grant you this scope. In such cases, you might want to consider using the Gmail API as the backend instead.
With IMAP and SMTP, Gmail counts data traffic and the number of email recipients. These limit counters are account-wide, meaning that if multiple email clients are connected to the account and collectively download a large amount of data or send emails to many recipients, they might hit the daily data transfer limit. When this limit is reached, IMAP access is blocked for all email clients for that email account.
For Microsoft 365 (MS365) and Outlook, the required OAuth2 scopes are IMAP.AccessAsUser.All and SMTP.Send. Be aware that many MS365 organizations have the SMTP scope disabled by default. While a user can grant access to it, using SMTP may still fail. In this case, you might want to consider using the MS Graph API as the backend instead.
EmailEngine supports two indexing modes when dealing with IMAP accounts: the "full" indexer and the "fast" indexer.
The full indexer builds a comprehensive index of the email account, which is stored in Redis. EmailEngine uses this index to detect new, deleted, and updated emails (such as when an email is marked as seen or unseen). While this method provides the most features, the downside is that the index can consume a significant amount of space in Redis. The indexing process can also be slow when processing email accounts with a large number of emails. In some cases, indexing might take so much time that the IMAP session expires and is closed, causing EmailEngine to fail in properly indexing the emails on that account.
The fast indexer stores only minimal required information about the state of the email account. There is almost no storage overhead in Redis, and the indexing process is, as the name suggests, fast. The downside is that EmailEngine can only detect new emails but not deleted or modified emails with the fast indexer.
To select an indexer, you can set the preferred indexer in the Service configuration page:
You can also specify the preferred indexer by providing the imapIndexer property in the account data when adding a new email account to EmailEngine. If you have not set it, the default preferred indexer setting will be used.
Note: Once an indexer is set for an account, it cannot be changed directly. If you want to change the indexer, you need to use the Account Flush API to clear the existing index for that account. You can set the new indexer as part of the request so that once the existing index has been deleted, a new index is configured based on the provided input.
When using the Gmail API as the mail backend, EmailEngine does not open IMAP or SMTP sessions against Gmail and G Suite. Instead, it makes REST API requests to the Gmail API servers whenever it needs to perform an operation. EmailEngine normalizes all inputs and outputs, so when using EmailEngine's API, it should be transparent whether the backend is an IMAP server or the Gmail API.
To normalize some results, such as listing emails, EmailEngine might need to run multiple API requests instead of a single request, which can be slower.
Considerations:
OAuth2 Scope:
https://www.googleapis.com/auth/gmail.modify.Notifications:
gmail.modify is a "restricted" scope and requires a mandatory security review.https://www.googleapis.com/auth/pubsub scope must also be set up for EmailEngine to use.When using the Microsoft Graph API as the mail backend, EmailEngine does not open IMAP or SMTP sessions against Microsoft 365 (MS365) and Outlook/Hotmail accounts. Instead, it makes REST API requests to the Microsoft Graph API servers whenever it needs to perform an operation.
Considerations:
OAuth2 Scopes:
https://graph.microsoft.com/Mail.ReadWrite and https://graph.microsoft.com/Mail.Send.Currently, EmailEngine cannot use other protocols or backends. It does not support POP3, ActiveSync, or proprietary APIs of different email hosting providers.