NAME
mu - index and search e-mail messages stored in Maildirs
SYNOPSIS
mu index [options]
mu cleanup [options]
mu find [options] <search expression>
mu view <file> [<files>]
mu mkdir [options] <dir> [<dirs>]
mu [options]
DESCRIPTION
mu is a set of tools for dealing with e-mail messages in Maildirs.
Its main function is enable searching of e-mail messages. It does so by
scanning a Maildir directory tree and analyzing the e-mail messages
found. The results of this analysis are stored in a database, which can
then be queried.
In addition to indexing and searching, mu also offers functionality for
viewing messages and creating maildirs.
mu can be used from the command line, or can be integrated with e-mail
clients. This manpage contains examples of both.
COMMANDS
mu offers the following commands:
index for indexing (analyzing) the contents of your Maildirs, and
storing the information in a database
cleanup
for removing messages from the database for which there is no
longer a corresponding message file in the file system
find for finding messages in your database, using certain search
parameters (see below for details)
view for displaying e-mail messages
mkdir for creating Maildirs
The various commands are discussed in more detail below
GENERAL OPTIONS
mu offers a number of general options that apply to all commands:
-a, --muhome
causes mu to use an alternative directory to store and read its
database and logs. By default, ~/.mu is used.
-d, --debug
makes mu generate extra debug information, useful for debugging
the program itself. By default, debug information goes to the
log file, ~/.mu/mu.log. It can safely be deleted when mu is not
running. Note, with the debug option, the log file can grow
rather quickly. See the note on logging below.
-q, --quiet
causes mu not to output informational messages and progress
information to standard output, but only to the log file. Error
messages will still be sent to standard error. Note that mu
index is much faster with --quiet, so it is recommended you use
this option when using mu from scripts etc.
-e, --log-stderr
causes mu not to output all log messages to standard error, in
addition to sending them to the log file.
-v, --version
outputs the mu-version and copyright information.
-h, --help
list the various command line options, while --help-index,
--help-find and --help-all list only the options for one
command, or all of the commands.
THE INDEX COMMAND
Using the index command, you can index your Maildir directories, and
store the information in a Xapian database.
index understands Maildirs as defined by Dan Bernstein for qmail(7). In
addition, it understands recursive Maildirs (Maildirs within Maildirs),
Maildir++. It can also deal with VFAT-based Maildirs which use ’!’ as
the seperators instead of ’:’ as used by Tinymail/Modest and some other
e-mail programs.
E-mail messages which are not stored in something resembling a maildir
leaf directory (cur and new) are ignored.
Currently, symlinks are not followed.
If there is a file called .noindex in a directory, the contents of that
directory and all of its subdirectories will be ignored. This can be
useful to exclude certain directories from the indexing process, for
example directories with spam-messages.
The first run of mu index may take a few minutes if you have a lot of
mail (ten thousands of messages). Fortunately, such a full scan needs
to be done only once, after that it suffices to index the changes,
which goes much faster. Also note that a substantial amount of the time
goes to printing the progress information; if you turn that off (with
-q or --quiet), it goes a lot faster. See the ’Note on performance’
below for more information.
The optional phase two of the indexing-process is the removal of
messages from the database for which there is no longer a corresponding
file in the Maildir. If you do not want this, you can use -n,
--nocleanup.
When mu index catches on of the signals SIGINT, SIGHUP or SIGTERM
(e.g,, when you press Ctrl-C during the indexing process), it tries to
shutdown gracefully; it tries to save and commit data, and close the
database etc. If it receives another signal (e.g,, when pressing Ctrl-C
once more), mu index will terminate immediately.
Indexing options
-m, --maildir=<maildir>
starts searching at <maildir>. By default, mu uses whatever the
MAILDIR environment variable is set to; if that is not set, it
tries ~/Maildir . In either case, the path must be absolute.
Also please see the note on mixing sub-maildirs below.
-r, --reindex
re-index all mails, even ones that are already in the database.
-n, --nocleanup
disables the database cleanup that mu does by default after
indexing.
-y, --rebuild
clear all messages from the database before indexing. This is
effectively the same as removing the database. The difference
with --reindex is that --rebuild guarantees that after the
indexing has finished, there are no ’old’ messages in the
database anymore, which is not true with --reindex when indexing
only a part of messages (using --maildir). For this reason, it
is necessary to run mu index --rebuild when there is an upgrade
in the database format. mu index will issue a warning about
this.
-u, --autoupgrade automatically use -y, --empty
when mu notices that the database version is not up-to-date.
This option is for use in cron scripts and the like, so they
won’t require any user interaction, even when mu introduces a
new database vesion.
NOTE: It is probably not a good idea to run multiple instances of mu
index concurrently. No data loss should occur, but one or more
of the instances may experience errors due to database locks.
Also note that, before indexing is completed, searches for
messages may fail, even if they have already been indexed, as
some of the esssential database information will only be written
in batches during the indexing process.
Furthermore, it is not recommended tot mix maildirs and sub-
maildirs within the hierarchy in the same database; for example,
it’s better not to index both with --maildir=~/MyMaildir and
--maildir=~/MyMaildir/foo, as this may lead to unexpected
results when searching with the the ’maildir:’ search parameter
(see below).
A note on performance
As a non-scientific benchmark, a simple test on the authors machine (a
Thinkpad X61s laptop using Linux 2.6.31 and an ext3 file system) with
no existing database, and a maildir with 14,200 messages:
$ sudo sh -c ’sync && echo 3 > /proc/sys/vm/drop_caches’
$ time mu index --quiet
83.36s user 6.49s system 43% cpu 3:26.21 total
(about 69 messages per second)
A second run, which is the more typical use case when there is a database
already, goes much faster:
$ sudo sh -c ’sync && echo 3 > /proc/sys/vm/drop_caches’
$ time mu index --quiet
0.29s user 0.62s system 14% cpu 6.409 total
(about 2219 message per second)
Note that each of test flushes the caches first; a more common use case might
be to run mu index when new mail has arrived; the cache may stay
quite ’warm’ in that case:
$ time mu index --quiet
0.19s user 0.21s system 98% cpu 0.402 total
which is more than 35,0000 messages per second (there is some variance here,
but the author has not seen it getting under 30,0000 messages per second).
THE CLEANUP COMMAND
The cleanup command removes messages for which no corresponding file
can be found, from the database. Note that this is done automatically
when running mu index (unless --nocleanup was specified).
THE FIND COMMAND
The find command starts a search for messages in the database that
match the search pattern.
The search pattern is taken as a command line parameter. If the search
parameter consists of multiple parts (multiple command line parameters)
they are treated as if there were a logical AND between them.
If you want to make your own constructions (using AND, OR, NOT etc.,
you have to put quote them so mu can consider them as a unit.
mu relies on the Xapian database for its searching capabilities, so it
offers all the search functionality that Xapian offers; please refer
to:
http://xapian.org/docs/queryparser.html
All queries are logged in <mu-home>/mu.log.
The basic way to search a message is to type some words matching it, as
you would do in a search engine on the internet, ie.
mu find monkey banana
will find all message that have both ’monkey’ and ’banana’. Matching is
case-insensitive and somewhat intelligent, in that it tries to
recognize various forms of a word (such as plulars); this is all
courtesy of Xapina.
mu also recognizes prefixes for specific fields in a messages; for
example:
mu find subject:penguin
to find messages with have the word penguin in the subject field. You
can abbreviate subject: to just s:. Here is the full table of the
search fields and their abbreviations:
cc,c CC (Carbon-Copy) recipient
from,f Message sender
subject,s Message subject
to,t To: recipient
maildir,m Maildir
msgid,i Message-ID
The Maildir field describes the directory path starting after the
Maildir-base path, and before the /cur/ or /new/ part. So for example,
if there’s a message with the file name
~/Maildir/lists/running/cur/1234.213:2,, you could find it (and all the
other messages in the same maildir) with:
mu find maildir:/lists/running
Note the starting ’/’. If you want to match mails in the ’root’
maildir, you can do with a single ’/’:
mu find maildir:/
Find options
The find-command has various options that influence the way mu displays
the results. If you don’t specify anything, the defaults are
--fields="d f s", --sortfield=date and --descending.
-f, --fields=<fields>
specifies a string that determines which fields are shown in the
output. This string consists of a number of characters (such as
’s’ for subject or ’f’ for from), which will replace with the
actual field in the output. Fields that are not known will be
output as-is, allowing for some simple formatting.
For example:
mu find subject:snow --fields "d f s"
would list the date, subject and sender of all messages with
’snow’ in the their subject.
The table of replacement characters is superset of the list
mentions for search parameters:
t to: recipient
c cc: (Carbon-Copy) recipient
d Sent date of the message
f Message sender (from:)
g Message flags (flags)
l Full path to the message (location)
p Message priority (high, normal, low)
s Message subject
i Message-id
m maildir
The message-flags output is a string, consisting of zero or more
of the following characters.
D Draft Message
F Flagged
N New message (in new/ Maildir)
P Passed (’Handled’)
R Replied
S Seen
T Marked for deletion
a Has attachment
s Signed message
x Encrypted message
Note that these are theoretical flags, which may or may not be
actually in use.
-s, --sortfield =<field> and -z, --descending
specifies the field to sort the search results by, and the
direction. The following fields are supported:
cc,c CC (Carbon-Copy) recipient
date,d message sent date
from,f message sender
maildir,m maildir
msgid,i message id
prio,p message priority
subject,s message subject
to,t To:-recipient
Thus, for example, to sort messages by date, you could specify:
$ mu find fahrrad --fields "d f s" --sortfield=date --descending
Note, if you specify a sortfield, by default, they are sorted in
descending order (e.g., from lowest to highest). This is usually
a good choice, but for dates it may be more useful to sort in
the opposite direction.
-x, --xquery
shows the Xapian query corresponding to your search terms. This
is mostly useful for debugging.
-l, --linksdir =<dir> and -c, --clearlinks
output the results as a maildir with symbolic links to the found
messages. This enables easy integration with mail-clients (see
below for more information). mu will create the maildir if it
does not exist yet.
If you specify --clearlinks, all existing symlinks will be
cleared from the target maildir; this allows for re-use of the
same directory. An alternative would be to delete the target
directory before, but this has a big chance of accidentaly
removing something that should not be removed.
$ mu find grolsch --linksdir=~/Maildir/search --clearlinks
will store links to found messages in ~/Maildir/search. If the
directory does not exist yet, it will be created.
Note: when mu creates a Maildir for these links, it
automatically inserts a .noindex file, to exclude the directory
from mu index.
Example queries
Here are some simple examples of mu search queries; you can make many
more complicated queries using various logical operators, parentheses
and so on, but in the author’s experience, it’s usually faster to find
a message with a simple query just searching for some words.
Find all messages with both ’bee’ and ’bird’ (in any field)
$ mu find ’bee AND bird’
or shorter, because AND is implied:
$ mu find bee bird
Find all messages with either Frodo or Sam:
$ mu find ’Frodo OR Sam’
Find all messages with the ’wombat’ as subject, and ’capibara’
anywhere:
$ mu find subject:wombat capibara
Find all messages in the ’Archive’ folder from Fred:
$ mu find from:fred maildir:Archive
Integrating mu find with mail clients
mutt For mutt you can use the following in your muttrc; pressing the
F8 key will start a search, and F9 will take you to the results.
# mutt macros for mu
macro index <F8> "<shell-escape>mu find -c -l ~/Maildir/search " "mu find"
macro index <F9> "<change-folder-readonly>~/Maildir/search" "display mu find results"
Wanderlust
If you use Wanderlust for emacs, the following definitons can be
used; typing ’Q’ will start a query.
;; mu integration for Wanderlust
(defvar mu-wl-mu-program "mu")
(defvar mu-wl-search-folder "search")
(defun mu-wl-search ()
"search for messages with ‘mu’, and jump to the results"
(interactive)
(let* ((muexpr (read-string "Find messages matching: "))
(sfldr (concat elmo-maildir-folder-path "/"
mu-wl-search-folder))
(cmdline (concat mu-wl-mu-program " find "
"--clearlinks --linksdir=’" sfldr "’ "
muexpr)))
(= 0 (shell-command cmdline))))
(defun mu-wl-search-and-goto ()
"search and jump to the folder with the results"
(interactive)
(if (mu-wl-search)
(wl-summary-goto-folder-subr
(concat "." mu-wl-search-folder)
’force-update nil nil t)
(message "Query failed")))
THE VIEW COMMAND
With the view command, you can view one or more e-mail messages as they
are stored as files on disk. It does not require or use the Xapian
database.
Currently, the command shows some common headers (From:, To:, Cc:,
Subject: and Date:) and the plain-text body of the message.
THE MKDIR COMMAND
With the mkdir command, you can create new Maildirs with permissions
0755. For example,
mu mkdir tom dick harry
will create three Maildirs tom, dick and harry.
If the creation somehow fails, for safety reasons, no attempt is made
to remove any parts that were created.
FILES
By default, mu index stores its message database in ~/.mu/xapian; the
database has an embedded version number, and mu will automatically
update it when it notices a different version. This allows for
automatic updating of mu-versions, without the need to clear out any
old databases.
However, note that versions of mu before 0.7 used a different scheme,
which put the database in ~/.mu/xapian-<version>. These older databases
can safely be deleted. Starting from version 0.7, this manual cleanup
should no longer be needed.
By default, mu stores logs of its operations and queries in
~/.mu/mu.log. Upon startup, mu checks the size of this log file. If it
exceeds 1 MB, it will be moved to ~/.mu/mu.log.old, overwriting any
existing file of that name, and start with an empty ~/.mu/mu.log. This
scheme allows for continued use of mu without the need for any manual
maintenance of log files.
To store these files elsewhere from their default location, one can use
the --muhome option, as discussed in the GENERAL OPTIONS section.
ENVIRONMENT
As mentioned, mu index uses MAILDIR to find the user’s Maildir if it
has not been specified explicitly --maildir=<maildir>. If MAILDIR is
not set, mu index will try ~/Maildir.
BUGS
There probably are some; please report bugs when you find them:
http://code.google.com/p/mu0/issues/list
AUTHOR
Dirk-Jan C. Binnema <djcb@djcbsoftware.nl>
SEE ALSO
maildir(5)