OFFLINEIMAP(1) OfflineIMAP Manual OFFLINEIMAP(1)NAME
OfflineIMAP - Powerful IMAP/Maildir synchronization and reader support
SYNOPSISofflineimap [ -1 ] [ -P profiledir ] [ -a accountlist ] [ -c configfile
] [ -d debugtype[,...] ] [ -f foldername[,...] ] [ -k [sec‐
tion:]option=value ] [ -l filename ] [ -o ] [ -u interface ]
offlineimap-h | --help
DESCRIPTION
OfflineIMAP is a tool to simplify your e-mail reading. With
OfflineIMAP, you can read the same mailbox from multiple computers.
You get a current copy of your messages on each computer, and changes
you make one place will be visible on all other systems. For instance,
you can delete a message on your home computer, and it will appear
deleted on your work computer as well. OfflineIMAP is also useful if
you want to use a mail reader that does not have IMAP support, has poor
IMAP support, or does not provide disconnected operation.
OfflineIMAP is FAST; it synchronizes my two accounts with over 50 fold‐
ers in 3 seconds. Other similar tools might take over a minute, and
achieve a less-reliable result. Some mail readers can take over 10
minutes to do the same thing, and some don't even support it at all.
Unlike other mail tools, OfflineIMAP features a multi-threaded synchro‐
nization algorithm that can dramatically speed up performance in many
situations by synchronizing several different things simultaneously.
OfflineIMAP is FLEXIBLE; you can customize which folders are synced via
regular expressions, lists, or Python expressions; a versatile and com‐
prehensive configuration file is used to control behavior; two user
interfaces are built-in; fine-tuning of synchronization performance is
possible; internal or external automation is supported; SSL and PREAUTH
tunnels are both supported; offline (or "unplugged") reading is sup‐
ported; and esoteric IMAP features are supported to ensure compatibil‐
ity with the widest variety of IMAP servers.
OfflineIMAP is SAFE; it uses an algorithm designed to prevent mail loss
at all costs. Because of the design of this algorithm, even program‐
ming errors should not result in loss of mail. I am so confident in
the algorithm that I use my own personal and work accounts for testing
of OfflineIMAP pre-release, development, and beta releases. Of course,
legally speaking, OfflineIMAP comes with no warranty, so I am not
responsible if this turns out to be wrong.
METHOD OF OPERATION
OfflineIMAP traditionally operates by maintaining a hierarchy of mail
folders in Maildir format locally. Your own mail reader will read mail
from this tree, and need never know that the mail comes from IMAP.
OfflineIMAP will detect changes to the mail folders on your IMAP server
and your own computer and bi-directionally synchronize them, copying,
marking, and deleting messages as necessary.
With OfflineIMAP 4.0, a powerful new ability has been introduced -- the
program can now synchronize two IMAP servers with each other, with no
need to have a Maildir layer in-between. Many people use this if they
use a mail reader on their local machine that does not support
Maildirs. People may install an IMAP server on their local machine,
and point both OfflineIMAP and their mail reader of choice at it. This
is often preferable to the mail reader's own IMAP support since
OfflineIMAP supports many features (offline reading, for one) that most
IMAP-aware readers don't. However, this feature is not as time-tested
as traditional syncing, so my advice is to stick with normal methods of
operation for the time being.
QUICK START
If you have already installed OfflineIMAP system-wide, or your system
administrator has done that for you, your task for setting up
OfflineIMAP for the first time is quite simple. You just need to set
up your configuration file, make your folder directory, and run it!
You can quickly set up your configuration file. The distribution
includes a file offlineimap.conf.minimal (see
/usr/share/doc/offlineimap-*/offlineimap.conf*) that is a basic example
of setting of OfflineIMAP. You can simply copy this file into your
home directory and name it .offlineimaprc (note the leading period). A
command such as cp offlineimap.conf.minimal ~/.offlineimaprc will do
it. Or, if you prefer, you can just copy this text to
~/.offlineimaprc:
[general]
accounts = Test
[Account Test]
localrepository = Local
remoterepository = Remote
[Repository Local]
type = Maildir
localfolders = ~/Test
[Repository Remote]
type = IMAP
remotehost = examplehost
remoteuser = jgoerzen
Now, edit the ~/.offlineimaprc file with your favorite editor. All you
have to do is specify a directory for your folders to be in (on the
localfolders line), the host name of your IMAP server (on the remote‐
host line), and your login name on the remote (on the remoteuser line).
That's it!
To run OfflineIMAP, you just have to say offlineimap-- it will fire
up, ask you for a login password if necessary, synchronize your fold‐
ers, and exit. See? You can just throw away the rest of this finely-
crafted, perfectly-honed manual! Of course, if you want to see how you
can make OfflineIMAP FIVE TIMES FASTER FOR JUST $19.95 (err, well, $0),
you have to read on!
INSTALLATION
If you are reading this document via the "man" command, it is likely
that you have no installation tasks to perform; your system administra‐
tor has already installed it. If you need to install it yourself, you
have options: system-wide installation with other systems, and a sin‐
gle-user installation. You can download the latest version of
OfflineIMAP from the OfflineIMAP website <URL:http://software.com‐
plete.org/offlineimap/>.
PREREQUISITES
In order to use OfflineIMAP, you need to have these conditions satis‐
fied:
· Your mail server must support IMAP. Most Internet Service Providers
and corporate networks do, and most operating systems have an IMAP
implementation readily available. A special Gmail mailbox type is
available to interface with Gmail's IMAP front-end.
· You must have Python version 2.4 or above installed. If you do not
have Python already, check with your system administrator or operat‐
ing system vendor; or, download it from the Python website
<URL:http://www.python.org/>. If you intend to use the SSL inter‐
face, your Python must have been built with SSL support.
· Have a mail reader that supports the Maildir mailbox format. Most
modern mail readers have this support built-in, so you can choose
from a wide variety of mail servers. This format is also known as
the "qmail" format, so any mail reader compatible with it will work
with OfflineIMAP. If you do not have a mail reader that supports
Maildir, you can often install a local IMAP server and point both
OfflineIMAP and your mail reader at it.
SYSTEM-WIDE INSTALLATION, OTHER
Download the tar.gz version of the package from the website
<URL:http://software.complete.org/offlineimap/>. Then run these com‐
mands, making sure that you are the "root" user first:
tar -zxvf offlineimap_x.y.z.tar.gz
cd offlineimap-x.y.z
python2.2 setup.py install
On some systems, you will need to use python instead of python2.2.
Next, proceed to [XRef to CONFIGURATION] below. You will type
offlineimap to invoke the program.
SINGLE-ACCOUNT INSTALLATION
Download the tar.gz version of the package from the website
<URL:http://software.complete.org/offlineimap/>. Then run these com‐
mands:
tar -zxvf offlineimap_x.y.z.tar.gz
cd offlineimap-x.y.z
When you want to run OfflineIMAP, you will issue the cd command as
above and then type ./offlineimap.py; there is no installation step
necessary.
CONFIGURATION
OfflineIMAP is regulated by a configuration file that is normally
stored in ~/.offlineimaprc. OfflineIMAP ships with a file named
offlineimap.conf that you should copy to that location and then edit.
This file is vital to proper operation of the system; it sets every‐
thing you need to run OfflineIMAP. Full documentation for the configu‐
ration file is included within the sample file.
OfflineIMAP also ships a file named offlineimap.conf.minimal that you
can also try. It's useful if you want to get started with the most
basic feature set, and you can read about other features later with
offlineimap.conf.
OPTIONS
Most configuration is done via the configuration file. Nevertheless,
there are a few command-line options that you may set for OfflineIMAP.
-1 Disable most multithreading operations and use solely a single-
connection sync. This effectively sets the maxsyncaccounts and
all maxconnections configuration file variables to 1.
-P profiledir
Sets OfflineIMAP into profile mode. The program will create
profiledir (it must not already exist). As it runs, Python pro‐
filing information about each thread is logged into profiledir.
Please note: This option is present for debugging and optimiza‐
tion only, and should NOT be used unless you have a specific
reason to do so. It will significantly slow program perfor‐
mance, may reduce reliability, and can generate huge amounts of
data. You must use the -1 option when you use -P.
-a accountlist
Overrides the accounts option in the general section of the con‐
figuration file. You might use this to exclude certain
accounts, or to sync some accounts that you normally prefer not
to. Separate the accounts by commas, and use no embedded spa‐
ces.
-c configfile
Specifies a configuration file to use in lieu of the default,
~/.offlineimaprc.
-d debugtype[,...]
Enables debugging for OfflineIMAP. This is useful if you are
trying to track down a malfunction or figure out what is going
on under the hood. I suggest that you use this with -1 to make
the results more sensible.
-d requires one or more debugtypes, separated by commas. These
define what exactly will be debugged, and include three options:
imap, maildir, and thread. The imap option will enable IMAP
protocol stream and parsing debugging. Note that the output may
contain passwords, so take care to remove that from the debug‐
ging output before sending it to anyone else. The maildir
option will enable debugging for certain Maildir operations.
And thread will debug the threading model.
-f foldername[,foldername]
Only sync the specified folders. The foldernames are the
untranslated foldernames. This command-line option overrides
any folderfilter and folderincludes options in the configuration
file.
-k [section:]option=value
Override configuration file option. If "section" is omitted, it
defaults to general. Any underscores "_" in the section name
are replaced with spaces: for instance, to override option
autorefresh in the "[Account Personal]" section in the config
file one would use "-k Account_Personal:autorefresh=30".
-l filename
Enables logging to filename. This will log everything that goes
to the screen to the specified file. Additionally, if any
debugging is specified with -d, then debug messages will not go
to the screen, but instead to the logfile only.
-o Run only once, ignoring all autorefresh settings in the configu‐
ration file.
-q Run only quick synchronizations. Ignore any flag updates on
IMAP servers.
-h
--help Show summary of options.
-u interface
Specifies an alternative user interface module to use. This
overrides the default specified in the configuration file. The
pre-defined options are listed in the User Interfaces section.
USER INTERFACES
OfflineIMAP has a pluggable user interface system that lets you choose
how the program communicates information to you. There are two graphi‐
cal interfaces, two terminal interfaces, and two noninteractive inter‐
faces suitable for scripting or logging purposes. The ui option in the
configuration file specifies user interface preferences. The -u com‐
mand-line option can override the configuration file setting. The
available values for the configuration file or command-line are
described in this section.
CURSES.BLINKENLIGHTS
Curses.Blinkenlights is an interface designed to be sleek, fun to
watch, and informative of the overall picture of what OfflineIMAP is
doing. I consider it to be the best general-purpose interface in
OfflineIMAP.
Curses.Blinkenlights contains a row of "LEDs" with command buttons and
a log. The log shows more detail about what is happening and is
color-coded to match the color of the lights.
Each light in the Blinkenlights interface represents a thread of execu‐
tion -- that is, a particular task that OfflineIMAP is performing right
now. The colors indicate what task the particular thread is perform‐
ing, and are as follows:
Black indicates that this light's thread has terminated; it will light
up again later when new threads start up. So, black indicates
no activity.
Red (Meaning 1)
is the color of the main program's thread, which basically does
nothing but monitor the others. It might remind you of HAL 9000
in 2001.
Gray indicates that the thread is establishing a new connection to
the IMAP server.
Purple is the color of an account synchronization thread that is moni‐
toring the progress of the folders in that account (not generat‐
ing any I/O).
Cyan indicates that the thread is syncing a folder.
Green means that a folder's message list is being loaded.
Blue is the color of a message synchronization controller thread.
Orange indicates that an actual message is being copied. (We use fuch‐
sia for fake messages.)
Red (meaning 2)
indicates that a message is being deleted.
Yellow / bright orange
indicates that message flags are being added.
Pink / bright red
indicates that message flags are being removed.
Red / Black Flashing
corresponds to the countdown timer that runs between synchro‐
nizations.
The name of this interfaces derives from a bit of computer history.
Eric Raymond's Jargon File defines blinkenlights, in part, as:
Front-panel diagnostic lights on a computer, esp. a dinosaur.
Now that dinosaurs are rare, this term usually refers to status
lights on a modem, network hub, or the like.
This term derives from the last word of the famous blackletter-
Gothic sign in mangled pseudo-German that once graced about half
the computer rooms in the English-speaking world. One version
ran in its entirety as follows:
ACHTUNG! ALLES LOOKENSPEEPERS!
Das computermachine ist nicht fuer gefingerpoken und mitten‐
grabben. Ist easy schnappen der springenwerk, blowenfusen und
poppencorken mit spitzensparken. Ist nicht fuer gewerken bei
das dumpkopfen. Das rubbernecken sichtseeren keepen das cotten-
pickenen hans in das pockets muss; relaxen und watchen das
blinkenlichten.
TTY.TTYUI
TTY.TTYUI interface is for people running in basic, non-color termi‐
nals. It prints out basic status messages and is generally friendly to
use on a console or xterm.
NONINTERACTIVE.BASIC
Noninteractive.Basic is designed for situations in which OfflineIMAP
will be run non-attended and the status of its execution will be
logged. You might use it, for instance, to have the system run auto‐
matically and e-mail you the results of the synchronization. This user
interface is not capable of reading a password from the keyboard;
account passwords must be specified using one of the configuration file
options.
NONINTERACTIVE.QUIET
Noninteractive.Quiet is designed for non-attended running in situations
where normal status messages are not desired. It will output nothing
except errors and serious warnings. Like Noninteractive.Basic, this
user interface is not capable of reading a password from the keyboard;
account passwords must be specified using one of the configuration file
options.
MACHINE.MACHINEUI
Machine.MachineUI generates output in a machine-parsable format. It is
designed for other programs that will interface to OfflineIMAP.
EXAMPLES
Here are some example configurations for various situations. Please e-
mail any other examples you have that may be useful to me.
MULTIPLE ACCOUNTS WITH MUTT
This example shows you how to set up OfflineIMAP to synchronize multi‐
ple accounts with the mutt mail reader.
Start by creating a directory to hold your folders by running mkdir
~/Mail. Then, in your ~/.offlineimaprc, specify:
accounts = Personal, Work
Make sure that you have both an [Account Personal] and an [Account
Work] section. The local repository for each account must have differ‐
ent localfolder path names. Also, make sure to enable [mbnames].
In each local repository section, write something like this:
localfolders = ~/Mail/Personal
Finally, add these lines to your ~/.muttrc:
source ~/path-to-mbnames-muttrc-mailboxes
folder-hook Personal set from="youremail@personal.com"
folder-hook Work set from="youremail@work.com"
set mbox_type=Maildir
set folder=$HOME/Mail
spoolfile=+Personal/INBOX
That's it!
UW-IMAPD AND REFERENCES
Some users with a UW-IMAPD server need to use OfflineIMAP's "reference"
feature to get at their mailboxes, specifying a reference of "~/Mail"
or "#mh/" depending on the configuration. The below configuration from
(originally from docwhat@gerf.org) shows using a reference of Mail, a
nametrans that strips the leading Mail/ off incoming folder names, and
a folderfilter that limits the folders synced to just three.
[Account Gerf]
localrepository = GerfLocal
remoterepository = GerfRemote
[Repository GerfLocal]
type = Maildir
localfolders = ~/Mail
[Repository GerfRemote]
type = IMAP
remotehost = gerf.org
ssl = yes
remoteuser = docwhat
reference = Mail
# Trims off the preceeding Mail on all the folder names.
nametrans = lambda foldername: \
re.sub('^Mail/', '', foldername)
# Yeah, you have to mention the Mail dir, even though it
# would seem intuitive that reference would trim it.
folderfilter = lambda foldername: foldername in [
'Mail/INBOX',
'Mail/list/zaurus-general',
'Mail/list/zaurus-dev',
]
maxconnections = 1
holdconnectionopen = no
PYTHONFILE CONFIGURATION FILE OPTION
You can have OfflineIMAP load up a Python file before evaluating the
configuration file options that are Python expressions. This example
is based on one supplied by Tommi Virtanen for this feature.
In ~/.offlineimaprc, he adds these options:
[general]
pythonfile=~/.offlineimap.py
[Repository foo]
foldersort=mycmp
Then, the ~/.offlineimap.py file will contain:
prioritized = ['INBOX', 'personal', 'announce', 'list']
def mycmp(x, y):
for prefix in prioritized:
xsw = x.startswith(prefix)
ysw = y.startswith(prefix)
if xsw and ysw:
return cmp(x, y)
elif xsw:
return -1
elif ysw:
return +1
return cmp(x, y)
def test_mycmp():
import os, os.path
folders=os.listdir(os.path.expanduser('~/data/mail/tv@hq.yok.utu.fi'))
folders.sort(mycmp)
print folders
This code snippet illustrates how the foldersort option can be custom‐
ized with a Python function from the pythonfile to always synchronize
certain folders first.
SIGNALS
OfflineIMAP writes its current PID into ~/.offlineimap/pid when it is
running. It is not guaranteed that this file will not exist when
OfflineIMAP is not running.
ERRORS
If you get one of some frequently-encountered or confusing errors,
please check this section.
UID VALIDITY PROBLEM FOR FOLDER
IMAP servers use a unique ID (UID) to refer to a specific message.
This number is guaranteed to be unique to a particular message forever.
No other message in the same folder will ever get the same UID. UIDs
are an integral part of OfflineIMAP's synchronization scheme; they are
used to match up messages on your computer to messages on the server.
Sometimes, the UIDs on the server might get reset. Usually this will
happen if you delete and then recreate a folder. When you create a
folder, the server will often start the UID back from 1. But
OfflineIMAP might still have the UIDs from the previous folder by the
same name stored. OfflineIMAP will detect this condition and skip the
folder. This is GOOD, because it prevents data loss.
You can fix it by removing your local folder and cache data. For
instance, if your folders are under ~/Folders and the folder with the
problem is INBOX, you'd type this:
rm -r ~/Folders/INBOX
rm -r ~/.offlineimap/Account-AccountName/LocalStatus/INBOX
rm -r ~/.offlineimap/Repository-RemoteRepositoryName/FolderValidity/INBOX
(Of course, replace AccountName and RemoteRepositoryName with the names
as specified in ~/.offlineimaprc).
Next time you run OfflineIMAP, it will re-download the folder with the
new UIDs. Note that the procedure specified above will lose any local
changes made to the folder.
Some IMAP servers are broken and do not support UIDs properly. If you
continue to get this error for all your folders even after performing
the above procedure, it is likely that your IMAP server falls into this
category. OfflineIMAP is incompatible with such servers. Using
OfflineIMAP with them will not destroy any mail, but at the same time,
it will not actually synchronize it either. (OfflineIMAP will detect
this condition and abort prior to synchronization.)
This question comes up frequently on the OfflineIMAP mailing list
<URL:http://lists.complete.org/offlineimap@complete.org/>. You can
find a detailed discussion <URL:http://lists.com‐
plete.org/offlineimap@complete.org/2003/04/msg00012.html.gz> of the
problem there.
CONFORMING TO
· Internet Message Access Protocol version 4rev1 (IMAP 4rev1) as speci‐
fied in RFC2060 and RFC3501
· CRAM-MD5 as specified in RFC2195
· Maildir as specified in the Maildir manpage
<URL:http://www.qmail.org/qmail-manual-html/man5/maildir.html> and
the qmail website <URL:http://cr.yp.to/proto/maildir.html>.
· Standard Python 2.2.1 as implemented on POSIX-compliant systems.
NOTES
DELETING LOCAL FOLDERS
OfflineIMAP does a two-way synchronization. That is, if you make a
change to the mail on the server, it will be propagated to your local
copy, and vise-versa. Some people might think that it would be wise to
just delete all their local mail folders periodically. If you do this
with OfflineIMAP, remember to also remove your local status cache
(~/.offlineimap by default). Otherwise, OfflineIMAP will take this as
an intentional deletion of many messages and will interpret your action
as requesting them to be deleted from the server as well. (If you
don't understand this, don't worry; you probably won't encounter this
situation.)
MULTIPLE INSTANCES
OfflineIMAP is not designed to have several instances (for instance, a
cron job and an interactive invocation) run over the same mailbox
simultaneously. It will perform a check on startup and abort if
another OfflineIMAP is already running. If you need to schedule syn‐
chronizations, you'll probably find autorefresh settings more conve‐
nient than cron. Alternatively, you can set a separate metadata direc‐
tory for each instance.
COPYING MESSAGES BETWEEN FOLDERS
Normally, when you copy a message between folders or add a new message
to a folder locally, OfflineIMAP will just do the right thing. How‐
ever, sometimes this can be tricky -- if your IMAP server does not pro‐
vide the SEARCH command, or does not return something useful,
OfflineIMAP cannot determine the new UID of the message. So, in these
rare instances, OfflineIMAP will upload the message to the IMAP server
and delete it from your local folder. Then, on your next sync, the
message will be re-downloaded with the proper UID. OfflineIMAP makes
sure that the message was properly uploaded before deleting it, so
there should be no risk of data loss.
MAILING LIST
There is an OfflineIMAP mailing list available. To subscribe, send the
text "Subscribe" in the subject of a mail to offlineimap-request@com‐
plete.org. To post, send the message to offlineimap@complete.org. Ar‐
chives are available at
<URL:http://lists.complete.org/offlineimap@complete.org/>.
BUGS
Reports of bugs should be reported online at the OfflineIMAP homepage.
UPGRADING TO 4.0
If you are upgrading from a version of OfflineIMAP prior to 3.99.12,
you will find that you will get errors when OfflineIMAP starts up
(relating to ConfigParser or AccountHashGenerator) and the configura‐
tion file. This is because the config file format had to change to
accommodate new features in 4.0. Fortunately, it's not difficult to
adjust it to suit.
First thing you need to do is stop any running OfflineIMAP instance,
making sure first that it's synced all your mail. Then, modify your
~/.offlineimaprc file. You'll need to split up each account section
(make sure that it now starts with "Account ") into two Repository sec‐
tions (one for the local side and another for the remote side.) See
the files offlineimap.conf.minimal and offlineimap.conf in the distri‐
bution if you need more assistance.
OfflineIMAP's status directory area has also changed. Therefore, you
should delete everything in ~/.offlineimap as well as your local mail
folders.
When you start up OfflineIMAP 4.0, it will re-download all your mail
from the server and then you can continue using it like normal.
COPYRIGHT
OfflineIMAP, and this manual, are Copyright (C) 2002 - 2006 John
Goerzen.
This program is free software; you can redistribute it and/or modify it
under the terms of the GNU General Public License as published by the
Free Software Foundation; either version 2 of the License, or (at your
option) any later version.
This program is distributed in the hope that it will be useful, but
WITHOUT ANY WARRANTY; without even the implied warranty of MER‐
CHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General
Public License for more details.
You should have received a copy of the GNU General Public License along
with this program; if not, write to the Free Software Foundation, Inc.,
59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
imaplib.py comes from the Python dev tree and is licensed under the
GPL-compatible PSF license as stated in the file COPYRIGHT in the
OfflineIMAP distribution.
AUTHOR
OfflineIMAP, its libraries, documentation, and all included files,
except where noted, was written by John Goerzen <jgoerzen@complete.org>
and copyright is held as stated in the COPYRIGHT section.
OfflineIMAP may be downloaded, and information found, from its homepage
<URL:http://software.complete.org/offlineimap>.
SEE ALSOmutt(1), python(1)HISTORY
Detailed history may be found in the file ChangeLog in the OfflineIMAP
distribution. This section provides a large overview.
Development on OfflineIMAP began on June 18, 2002. Version 1.0.0 was
released three days later on June 21, 2002. Point releases followed,
including speed optimizations and some compatibility fixes.
Version 2.0.0 was released on July 3, 2002, and represented the first
time the synchronization became multithreaded and, to the best of my
knowledge, the first multithreaded IMAP syncrhonizing application in
existance. The last 2.0.x release, 2.0.8, was made on July 9.
Version 3.0.0 was released on July 11, 2002, and introduced modular
user interfaces and the first GUI interface for OfflineIMAP. This man‐
ual also was introduced with 3.0.0, along with many command-line
options. Version 3.1.0 was released on July 21, adding the Noninterac‐
tive user interfaces, profiling support, and several bugfixes. 3.2.0
was released on July 24, adding support for the Blinkenlights GUI
interface. OfflineIMAP entered maintenance mode for awhile, as it had
reached a feature-complete milestone in my mind.
The 3.99.x branch began in on October 7, 2002, to begin work for 4.0.
The Curses.Blinkenlights interface was added in 3.99.6, and many archi‐
tectural changes were made.
4.0.0 was released on July 18, 2003, including the ability to synchro‐
nize directly between two IMAP servers, the first re-architecting of
the configuration file to refine the notion of an account, and the new
Curses interface.
John Goerzen 17 November 2009 OFFLINEIMAP(1)
