Automx2

Automx2 sets up a service that allows various email clients to automatically configure themselves on the basis of information delivered through a webpage.

https://rseichter.github.io/automx2/ setup guide

https://github.com/rseichter/automx2 actual github page

https://erlangen-sheppy.medium.com/client-auto-config-for-self-hosted-mail-8947c16f77d5 - useful guide

https://www.ietf.org/archive/id/draft-ietf-mailmaint-autoconfig-00.html - spec

Initial install

Gotchas during setup

When setting up sudo to the automx2 user. Not for setting up the service in systemd

After the first run (as automx2 user)

~/.venv/bin/flask.sh h run --host=127.0.0.1 --port=4243

use a different terminal to run the tests.

Don't use the json file to populate the database, it's not very useful.

populating the database

Change to sqlite database and test population

Now in ~/.automx2.conf comment out db_uri = sqlite:///:memory:

and set the db line to

db_uri = sqlite:////var/www/automx2/db.sqlite

Then when you populate the database it will create the file.

It is possible to populate the database with the json file, but then do not delete any of the lines of the example! It's just as good (if not better) to populate using the

curl -X GET http://127.0.0.1:4243/initdb/

command as it gets you more sane input.

You can see what it gives you if you then run (if you have libxml2-utils installed)

curl 'http://127.0.0.1:4243/mail/config-v1.1.xml?emailaddress=user@example.com' 2>/dev/null | xmllint --format -

<?xml version="1.0"?>
<clientConfig version="1.1">
  <emailProvider id="automx2-1000">
    <identity/>
    <domain>example.com</domain>
    <domain>example.net</domain>
    <domain>example.org</domain>
    <displayName>Big Corporation, Inc.</displayName>
    <displayShortName>BigCorp</displayShortName>
    <outgoingServer type="smtp">
      <hostname>primary-smtp.ca10d6201c36438dae4038d3c30b9731.com</hostname>
      <port>587</port>
      <socketType>STARTTLS</socketType>
      <username>%EMAILADDRESS%</username>
      <authentication>plain</authentication>
    </outgoingServer>
    <incomingServer type="imap">
      <hostname>imap1.6e457bf8fd81421c92df72ecff3ebf19.com</hostname>
      <port>143</port>
      <socketType>STARTTLS</socketType>
      <username>%EMAILADDRESS%</username>
      <authentication>plain</authentication>
    </incomingServer>
  </emailProvider>
</clientConfig>

NB you can delete everything in the database by using

curl -X DELETE http://127.0.0.1:4243/initdb/

Populating with the correct values

Then you can edit the entries in the database

sqlite3 ~automx2/db.sqlite

delete from davserver;
delete from davserver_domain;
delete from provider;
insert into provider values(1000, "Edgar BV", "EdgarBV");
delete from domain;
insert into domain values(3000, 'edgarbv.com', 1000, NULL);
delete from server;
insert into server values(4000, 10, "mail.edgarbv.com", 465, 'smtp', 'SSL', '%EMAILADDRESS%', 'plain');
insert into server values(4001, 10, "mail.edgarbv.com", 993, 'imap', 'SSL', '%EMAILADDRESS%', 'plain');
delete from server_domain;
insert into server_domain values(4000, 3000);
insert into server_domain values(4001, 3000);

NB you can add secondaries in the server tables as well

NNB you can also use STARTTLS instead of SSL as a socket type

NNNB to see the values for the davserver, see under mobileconfig (below)

Testing the populated database

Now when you test using

curl 'http://127.0.0.1:4243/mail/config-v1.1.xml?emailaddress=user@edgarbv.com' 2>/dev/null | xmllint --format -

You should see

<?xml version="1.0"?>
<clientConfig version="1.1">
  <emailProvider id="automx2-1000">
    <identity/>
    <domain>edgarbv.com</domain>
    <domain>flyrob.in</domain>
    <domain>ajanaku.com</domain>
    <displayName>Edgar BV</displayName>
    <displayShortName>EdgarBV</displayShortName>
    <outgoingServer type="smtp">
      <hostname>mail.edgarbv.com</hostname>
      <port>465</port>
      <socketType>SSL</socketType>
      <username>%EMAILADDRESS%</username>
      <authentication>plain</authentication>
    </outgoingServer>
    <incomingServer type="imap">
      <hostname>mail.edgarbv.com</hostname>
      <port>993</port>
      <socketType>SSL</socketType>
      <username>%EMAILADDRESS%</username>
      <authentication>plain</authentication>
    </incomingServer>
  </emailProvider>
</clientConfig>

starting the systemd service

as root /etc/systemd/system/automx2.service

[Unit]
After=network.target
Description=MUA configuration service
Documentation=https://rseichter.github.io/automx2/

[Service]
Environment=FLASK_APP=automx2.server:app
Environment=FLASK_CONFIG=production
# Change paths in this file as necessary, depending on your
# chosen method of installation.
ExecStart=/var/www/automx2/.venv/bin/flask run --host=127.0.0.1 --port=4243
NotifyAccess=exec
Restart=always
Type=notify
User=automx2
WorkingDirectory=/var/www/automx2

[Install]
WantedBy=multi-user.target

reload the services list

systemctl daemon-reload

the status should load in now

systemctl status automx2

now enable and run the service

systemctl enable automx2 --now

Configuring for apache

first change the loglevel by editing ~automx2/.automx.conf

loglevel = WARNING

/etc/apache2/sites-available# cat autoconfig.edgarbv.com.conf

<VirtualHost *:80>
    ServerName autoconfig.edgarbv.com
    ServerAlias autodiscover.* autoconfig.*
    ProxyPreserveHost On
    ProxyPass "/" "http://127.0.0.1:4243/"
    ProxyPassReverse "/" "http://127.0.0.1:4243/"
    <Location /initdb>
        # Limit access to clients connecting from localhost
        Order Deny,Allow
        Deny from all
        Allow from 127.0.0.1
    </Location>
    ErrorLog ${APACHE_LOG_DIR}/automx.error.log
    CustomLog ${APACHE_LOG_DIR}/automx.access.log combined
</VirtualHost>

/etc/apache2/sites-available# cat autoconfig.edgarbv.com-le-ssl.conf

<VirtualHost *:443>
    ServerName autoconfig.edgarbv.com
    ServerAlias autodiscover.* autoconfig.*
    SSLCertificateFile /etc/letsencrypt/live/autoconfig.edgarbv.com-0001/fullchain.pem
    SSLCertificateKeyFile /etc/letsencrypt/live/autoconfig.edgarbv.com-0001/privkey.pem
    Include /etc/letsencrypt/options-ssl-apache.conf
    ProxyPreserveHost On
    ProxyPass "/" "http://127.0.0.1:4243/"
    ProxyPassReverse "/" "http://127.0.0.1:4243/"
    <Location /initdb>
        # Limit access to clients connecting from localhost
        Order Deny,Allow
        Deny from all
        Allow from 127.0.0.1
    </Location>
    ErrorLog ${APACHE_LOG_DIR}/automx.error.log
    LogLevel warn
    CustomLog ${APACHE_LOG_DIR}/automx.access.log combined
</VirtualHost>

NB logfile names include error.log and access.log for fail2ban

You can then test port 80 like

curl 'http://autodiscover.edgarbv.com/mail/config-v1.1.xml?emailaddress=user@flyrob.in' 2>/dev/null | xmllint --format -

and port 443

curl 'https://autodiscover.edgarbv.com/mail/config-v1.1.xml?emailaddress=user@flyrob.in' 2>/dev/null | xmllint --format -

DNS / SOA settings for domains

add autodiscover (thunderbird) and autoconfig (Microsoft) A records

autodiscover IN A serverip

autoconfig IN A serverip

_autodiscover._tcp IN SRV 10 0 443 mail.edgarbv.com.

Adding more domains

To add more domains to the database you have to add them to the database

~$ sqlite3 db.sqlite

insert into domain values(3001, 'flyrob.in', 1000, NULL);
insert into server_domain values(4000, 3001);
insert into server_domain values(4001, 3001);
.quit

add Let's Encrypt Certificate

You have to recreate the whole certificate so add all the domains to the list and re-create

certbot certonly -d mail.edgarbv.com,autoconfig.edgarbv.com,autodiscover.edgarbv.com,autodiscover.ajanaku.com,autoconfig.ajanaku.com,autoconfig.flyrob.in,autodiscover.flyrob.in

choose option 1

IOS problems

NB added mail.edgarbv.com because of https://learn.microsoft.com/ms-my/exchange/troubleshoot/mobile-devices/cannot-create-mail-profile-on-ios-device-via-autodiscover for IOS which states "Make sure that the certificate matches the user's SMTP domain, or manually configure account settings for the Exchange Online mailbox."

Also see https://practical365.com/fixing-autodiscover-root-domain-lookup-issues-mobile-devices/ how to try to solve this

But this does not seem to help much.

Test with

curl 'http://127.0.0.1:4243/mail/config-v1.1.xml?emailaddress=user@flyrob.in' 2>/dev/null | xmllint --format -

Testing

Logging

root@edgarinet:~# systemctl stop automx2
root@edgarinet:~# su - automx2

~automx2/.automx2.conf change

loglevel = DEBUG (from WARNING)

Then run it in a console to monitor the output

~automx2/.venv/bin/flask.sh run --host=127.0.0.1 --port=4243 --debug

You can also tail /var/log/apache/automx.log and /var/log/apache/automx.err

test with outlook 2006

payload.xml

<Autodiscover xmlns="http://schemas.microsoft.com/exchange/autodiscover/outlook/requestschema/2006">
  <Request>
    <AcceptableResponseSchema>http://schemas.microsoft.com/exchange/autodiscover/outlook/responseschema/2006a</AcceptableResponseSchema>
    <EMailAddress>jdoe@edgarbv.com</EMailAddress>
  </Request>
</Autodiscover>

test with

curl -L -d @payload.xml -H "Content-Type: application/xml" 'http://127.0.0.1:4243/autodiscover/autodiscover.xml'

curl -L -d @payload.xml -H "Content-Type: application/xml" 'http://autodiscover.edgarbv.com/autodiscover/autodiscover.xml'

curl -L -d @payload.xml -H "Content-Type: application/xml" 'https://autodiscover.flyrob.in/autodiscover/autodiscover.xml

This makes sure the SSL certificate works, because outlook requires a valid (not self signed) certificate

Test for mobileconfig

 curl -X GET http://127.0.0.1:4243/mobileconfig/?emailaddress=user@edgarbv.com

Adding davserver configuration Above we deleted the entries for davserver and davserver_domain from the sqlite db. The example data written was

sqlite> select  * from davserver;
4100|https://caldav.9e64a1e3b1b04d61a165397b74eeae80.com|443|caldav|1|0|%EMAILADDRESS%
4101|http://carddav.f7cb575b33e047c093b2c853d61ce194.com|0|carddav|0|1|
sqlite> select * from davserver_domain;
4100|3000
4101|3000
4101|3001

Updating the software

make sure you are su automx2

cd /srv/www/automx2
.venv/bin/pip install --upgrade automx2