Grabber

The asynchronous mail grabber or asynchronous grabber is a feature that collects and processes emails from an IMAP server in multi-threaded mode.

See also

Read more about the asynchronous grabber

Attention

The grabber uses port 8081 for the technical web interface that displays its current status.

Software

Software

Version

Operating System

  • Ubuntu 20.04

  • Ubuntu 22.04

Additional Software

  • glibc=2.28+

  • Python 3.7

Check current Python version:

python3 --version

Install Python 3.7:

sudo apt install python3.7

Mail Server

The grabber works with any IMAP server, including:

  • Microsoft Exchange Server

  • Gmail

RAM

Per 1000 emails:

Email type

Processing time (hours)

RAM

Light

2.2

1 GB

Heavy

10.1

6 GB

Mixed

4.6

3 GB

Types of emails:

  • light emails: 1 MB

  • heavy emails: 50 MB

  • mixed: 15.7 MB, where 70% of emails are 1 MB and 30% are 50 MB

Attention

The table lists the minimum RAM requirements. They only take the mail grabber requirements into account and do not include operating system overhead, Python processes, or other software.

The RAM required for processing emails is calculated using the formula:

RAM = AVERAGE_MSG_SIZE * (WORKERS_DOWNLOADERS * DOWNLOAD_BATCH_SIZE + WORKERS_PARSERS + 2 * REPORT_BATCH_SIZE)

where:

  • AVERAGE_MSG_SIZE — average email size in megabytes, depends on email attachments

Parameters set in the grabber configuration file:

  • WORKERS_DOWNLOADERS — number of threads for downloading emails, default value: 5

  • DOWNLOAD_BATCH_SIZE — amount of emails downloaded from the IMAP server at a time, default value: 10

  • WORKERS_PARSERS — number of threads for processing emails, default value: 10

  • REPORT_BATCH_SIZE — amount of emails sent to Staffcop at a time, default value: 10

Note: If the average email size is 25 MB, 2000 additional MB of RAM will be required.

25 * (5 * 10 + 10 + 2 * 10) = 2000 MB

Last Updated: 01.12.25