202 lines
14 KiB
ReStructuredText
202 lines
14 KiB
ReStructuredText
|
How my e-mail gets to that other guy?
|
|||
|
#####################################
|
|||
|
|
|||
|
:date: 2015-08-27T21:47:19Z
|
|||
|
:category: blog
|
|||
|
:tags: technology
|
|||
|
:url: 2015/08/27/how-my-email-gets-to-that-other-guy/
|
|||
|
:save_as: 2015/08/27/how-my-email-gets-to-that-other-guy/index.html
|
|||
|
:status: published
|
|||
|
:author: Gergely Polonkai
|
|||
|
|
|||
|
A friend of mine asked me how it is possible that she pushes buttons on her keyboard and mouse,
|
|||
|
and in an instant her peer reads the text she had in her mind. This is a step-by-step
|
|||
|
introduction of what happens in-between.
|
|||
|
|
|||
|
From your mind to your computer
|
|||
|
===============================
|
|||
|
|
|||
|
When you decide to write an e-mail to an acquaintance of yours, you open up your mailing software
|
|||
|
(this document doesn’t cover using mail applications you access through your browsers, just plain
|
|||
|
old Thunderbird, Outlook or similar programs. However, it gets the same after the mail left your
|
|||
|
computer), and press the “New Mail” button. What happens during this process is not covered in
|
|||
|
this article, but feel free to ask me in a comment! Now that you have your Mail User Agent (MUA)
|
|||
|
up and running, you begin typing.
|
|||
|
|
|||
|
When you press a button on your keyboard or mouse, a bunch of bits gets through the wire (or
|
|||
|
through air, if you went wireless) and get into your computer. I guess you learned about Morse
|
|||
|
during school; imagine two `Morse operators
|
|||
|
<http://www.uscupstate.edu/academics/education/aam/lessons/susan_sawyer/morse%20code.jpg>`_, one
|
|||
|
in your keyboard/mouse, and one in your computer. Whenever you press a key, that tiny creature
|
|||
|
sends a series of short and long beeps (called 0 or 1 bits, respectively) to the operator in your
|
|||
|
computer (fun fact: have you ever seen someone typing at an amazing speed of 5 key presses per
|
|||
|
second? Now imagine that whenever that guy presses a key on their keyboard, that tiny little
|
|||
|
Morse operator pressing his button 16 times for each key press, with perfect timing so that the
|
|||
|
receiving operator can decide if that was a short or long beep.)
|
|||
|
|
|||
|
Now that the code got to the operator inside the machine, it’s up to him to decode it. The funny
|
|||
|
thing about keyboards and computers is that the computer doesn’t receive the message “Letter Q was
|
|||
|
pressed”, but instead “The second button on the second row was pressed” (a number called scan
|
|||
|
code). At this time the operator decodes this information (in this example it is most likely this
|
|||
|
Morse code: ``···-···· -··-····``) and checks one of his tables titled “Current Keyboard Layout.”
|
|||
|
It says this specific key corresponds to letter ‘Q’, so it forwards this information (I mean the
|
|||
|
letter; after this step your computer doesn’t care which plastic slab you hit, just the letter
|
|||
|
‘Q’) to your MUA, inserts it into the mail in its memory, then displaying it happily (more about
|
|||
|
this step later).
|
|||
|
|
|||
|
When you finish your letter you press the send button of your MUA. First it converts all the
|
|||
|
pretty letters and pictures to something a computer can understand (yes, those Morse codes, or
|
|||
|
more precisely, zeros and ones, again). Then it adds loads of meta data, like your name and
|
|||
|
e-mail address, the current date and time including the time zone and pass it to the sending parts
|
|||
|
of the MUA so the next step can begin.
|
|||
|
|
|||
|
IP addresses, DNS and protocols
|
|||
|
===============================
|
|||
|
|
|||
|
The Internet is a huge amount of computers connected with each other, all of them having at least
|
|||
|
one address called IP address that looks something like this: ``123.234.112.221``. These are four
|
|||
|
numbers between 0 and 255 inclusive, separated by dots. This makes it possible to have
|
|||
|
4,294,967,296 computers. With the rules of address assignment added, this is actually reduced to
|
|||
|
3,702,258,432; a huge number, still, but it is not enough, as in the era of the Internet of Things
|
|||
|
everything is interconnected, up to and possibly including your toaster. Thus, we are slowly
|
|||
|
transitioning to a new addressing scheme that looks like this:
|
|||
|
``1234:5678:90ab:dead:beef:9876:5432:1234``. This gives an enormous amount of
|
|||
|
340,282,366,920,938,463,463,374,607,431,768,211,456 addresses, with only
|
|||
|
4,325,185,976,917,036,918,000,125,705,034,137,602 of them being reserved, which gives us only a
|
|||
|
petty 335,957,180,944,021,426,545,374,481,726,734,073,854 available.
|
|||
|
|
|||
|
Imagine a large city with `that many buildings
|
|||
|
<http://www.digitallifeplus.com/wp-content/uploads/2012/07/new-york-city-aerial-5.jpg>`_, all of
|
|||
|
them having only a number: their IP address. No street names, no company names, no nothing. But
|
|||
|
people tend to be bad at memorizing numbers, so they started to give these buildings names. For
|
|||
|
example there is a house with the number ``216.58.209.165``, but between each other, people call
|
|||
|
it ``gmail.com``. Much better, isn’t it? Unfortunately, when computers talk, they only
|
|||
|
understand numbers so we have to provide them just that.
|
|||
|
|
|||
|
As remembering this huge number of addresses is a bit inconvenient, we created Domain Name
|
|||
|
Service, or DNS for short. A “domain name” usually (but not always) consist of two strings of
|
|||
|
letters, separated by dots (e.g. polonkai.eu, gmail.com, my-very-long-domain.co.uk, etc.), and a
|
|||
|
hostname is a domain name occasionally prefixed with something (e.g. **www**.gmail.com,
|
|||
|
**my-server**.my-very-long-domain.co.uk, etc.) One of the main jobs of DNS is to keep record of
|
|||
|
hostname/address pairs. When you enter ``gmail.com`` (which happens to be both a domain name and a
|
|||
|
hostname) in your browser’s address bar, your computer asks the DNS service if it knows the actual
|
|||
|
address of the building that people call ``gmail.com``. If it does, it will happily tell your
|
|||
|
computer the number of that building.
|
|||
|
|
|||
|
Another DNS job is to store some meta data about these domain names. For such meta data there are
|
|||
|
record types, one of these types being the Mail eXchanger, or MX. This record of a domain tells
|
|||
|
the world who is handling incoming mails for the specified domain. For ``gmail.com`` this is
|
|||
|
``gmail-smtp-in.l.google.com`` (among others; there can be multiple records of the same type, in
|
|||
|
which case they usually have priorities, too.)
|
|||
|
|
|||
|
One more rule: when two computers talk to each other they use so called protocols. These
|
|||
|
protocols define a set of rules on how they should communicate; this includes message formatting,
|
|||
|
special code words and such.
|
|||
|
|
|||
|
From your computer to the mail server
|
|||
|
=====================================
|
|||
|
|
|||
|
Your MUA has two settings called SMTP server address SMTP port number (see about that later).
|
|||
|
SMTP stands for Simple Mail Transfer Protocol, and defines the rules on how your MUA, or another
|
|||
|
mail handling computer should communicate with a mail handling computer when *sending* mail. Most
|
|||
|
probably your Internet Service Provider gave you an SMTP server name, like ``smtp.aol.com`` and a
|
|||
|
port number like ``587``.
|
|||
|
|
|||
|
When you hit that send button of yours, your computer will check with the DNS service for the
|
|||
|
address of the SMTP server, which, for ``smtp.aol.com``, is ``64.12.88.133``. The computer puts
|
|||
|
this name/address pair into its memory, so it doesn’t have to ask the DNS again (this technique is
|
|||
|
called caching and is widely used wherever time consuming operations happen).
|
|||
|
|
|||
|
Then it will send your message to the given port number of this newly fetched address. If you
|
|||
|
imagined computers as office buildings, you can imagine port numbers as departments and there can
|
|||
|
be 65535 of them in one building. The port number of SMTP is usually 25, 465 or 587 depending on
|
|||
|
many things we don’t cover here. Your MUA prepares your letter, adding your e-mail address and
|
|||
|
the recipients’, together with other information that may be useful for transferring your mail.
|
|||
|
It then puts this well formatted message in an envelope and writes “to building ``64.12.88.133``,
|
|||
|
dept. ``587``”, and puts it on the wire so it gets there (if the wire is broken, the building does
|
|||
|
not exist or there is no such department, you will get an error message from your MUA). Your
|
|||
|
address and the recipient’s address are inside the envelope; other than the MUA, your own computer
|
|||
|
is not concerned about it.
|
|||
|
|
|||
|
The mailing department (or instead lets call it the Mail Transfer Agent, A.K.A. MTA) now opens
|
|||
|
this envelope and reads the letter. All of it, letter by letter, checking if your MUA formatted
|
|||
|
it well. More than probably it also runs your message through several filters to decide if you
|
|||
|
are a bad guy sending some unwanted letter (also known as spam), but most importantly it fetches
|
|||
|
the recipients address. It is possible, e.g. when you send an e-mail within the same
|
|||
|
organization, that the recipient’s address is handled by this very same computer. In this case
|
|||
|
the MTA puts the mail to the recipient’s mailbox and the next step is skipped.
|
|||
|
|
|||
|
From one server to another
|
|||
|
==========================
|
|||
|
|
|||
|
Naturally, it is possible to send an e-mail from one company to another, so these MTAs don’t just
|
|||
|
wait for e-mails from you, but also communicate with each other. When you send a letter from your
|
|||
|
``example@aol.com`` address to me at ``gergely@polonkai.eu``, this is what happens.
|
|||
|
|
|||
|
In this case, the MTA that initially received the e-mail from you (which happened to be your
|
|||
|
Internet Service Provider’s SMTP server) turns to the DNS again. It will ask for the MX record of
|
|||
|
the domain name specified by the e-mail address, (the part after the ``@`` character, in my case,
|
|||
|
``polonkai.eu``), because the server mentioned that must be contacted, so they can deliver your
|
|||
|
mail for me. My domain is configured so its primary MX record is ``aspmx.l.google.com`` and the
|
|||
|
secondary is ``alt1.aspmx.l.google.com`` (and 5 more. Google likes to play it safe.) The MTA
|
|||
|
then gets the first server name, asks the DNS for its address, and tries to send a message to the
|
|||
|
``173.194.67.27`` (the address of ``aspmx.l.google.com``), same department. But unlike your MUA,
|
|||
|
MTAs don’t have a pre-defined port number for other MTAs (although there can be exceptions).
|
|||
|
Instead, they use well-known port numbers, ``465`` and ``25``. If the MTA on that server cannot
|
|||
|
be contacted for any reason, it tries the next one on the list of MX records. If none of the
|
|||
|
servers can be contacted, it will retry based on a set of rules defined by the administrators,
|
|||
|
which usually means it will retry after 1, 4, 24 and 48 hours. If there is still no answer after
|
|||
|
that many attempts, you will get an error message back, in the form of an e-mail sent directly by
|
|||
|
the SMTP server.
|
|||
|
|
|||
|
Once the other MTA could be contacted, your message is sent there. The original envelope you used
|
|||
|
is discarded, and a new one is used with the address and dept. number (port) of the receiving MTA.
|
|||
|
Also, your message gets altered a little bit, as most MTAs are kind enough (ie. not sneaky) to add
|
|||
|
a clause to your message stating “the MTA at <organization> has checked and forwarded this
|
|||
|
message.”
|
|||
|
|
|||
|
It is possible, though not likely, that your message gets through more than two MTAs (one at your
|
|||
|
ISP and one at the receiver’s) before arriving to its destination. At the end, an MTA will say
|
|||
|
that “OK, this recipient address is handled by me”, your message stops and stays there, put in
|
|||
|
your peer’s mailbox.
|
|||
|
|
|||
|
The mailbox
|
|||
|
-----------
|
|||
|
|
|||
|
Now that the MTA has passed your mail to the mailbox team (I call it a team instead of department
|
|||
|
because the tasks described here are usually handled by the MTA, too), it reads it. (Pesky little
|
|||
|
guys these mail handling departments, aren’t they?) If the mailbox has some filtering rules, like
|
|||
|
“if XY sends me a letter, mark it as important” or “if the letter has a specific word in its
|
|||
|
subject, put it in the XY folder”, it executes them, but the main point is to land the message in
|
|||
|
the actual post box of the recipient.
|
|||
|
|
|||
|
From the post box to the recipients computer
|
|||
|
============================================
|
|||
|
|
|||
|
When the recipient opens their MUA, it will look to a setting usually called “Incoming mail
|
|||
|
server”. Just like the SMTP server, it has a name and port number, along with a server type.
|
|||
|
This type can vary from provider to provider, and is usually one of POP3 (pretty old protocol,
|
|||
|
doesn’t even support folders on its own), IMAP (a newer one, with folders and message flags like
|
|||
|
“important”), MAPI (a dialect of IMAP, created by Microsoft as far as I know), or plain old mbox
|
|||
|
files on the receiving computer (this last option is pretty rare nowadays, so I don’t cover this
|
|||
|
option. Also, if you use these, you most probably don’t really need this article to understand
|
|||
|
how these things work.) This latter setting defines the protocol, telling your MUA how to “speak”
|
|||
|
to the post box.
|
|||
|
|
|||
|
So your MUA turns to the DNS once more to get the address of your incoming mail server and
|
|||
|
contacts it, using the protocol set by the server type. At the end, the recipients computer will
|
|||
|
receive a bunch of envelopes including the one that contains your message. The MUA opens them one
|
|||
|
by one and reads them, making a list ordered by their sender or subject, or the date of sending.
|
|||
|
|
|||
|
From the recipient’s comupter to their eyes
|
|||
|
===========================================
|
|||
|
|
|||
|
When the recipient then clicks on one of these mails, the MUA will fetch all the relevant bits
|
|||
|
like the sender, the subject line, the date of sending and the contents itself and sends it to the
|
|||
|
“printing” department (I use quotes as they don’t really print your mail on paper, they just
|
|||
|
convert it to a nice image so the recipient can see it. This is sometimes referred to as a
|
|||
|
rendering engine). Based on a bunch of rules they pretty-print it and send it to your display as
|
|||
|
a new series of Morse codes. Your display then decides how it will present it to the user: draw
|
|||
|
the pretty pictures if it is a computer screen, or just raise and lower some hard dots that
|
|||
|
represents letters on a Braille terminal.
|