Installing a new webserver
Basic Debian and network setup
apt-get install mysql-server openssh-server proftpd apache2 libapache2-mod-perl2 libapache2-mod-php5 php5 php5-cli php5-ffmpeg php5-gd php5-imagick php5-mcrypt php5-mhash php5-mysql php5-xmlrpc php5-xsl php5-curl php-apc snmp snmpd iotop mtop apachetop iptstate atsar postfix popa3d shorewall vim awstats bmon nscd sshfs mc zip unzip bzip2 arj spamassassin pyzor razor ncftp rsync phpmyadmin quota ntpdate ntp vacation pdnsd
Also ensure backports are enabled by adding
deb http://ftp.debian.org/debian jessie-backports main
to /etc/apt/sources.list
Then
sudo apt-get install python-certbot-apache -t jessie-backports
For Letsencrypt.
pdnsd - use resolvconf configuration
Set up networking in /etc/network/interfaces
# The loopback network interface
auto lo
iface lo inet loopback
# The primary network interface
allow-hotplug eth0
#auto eth0
iface eth0 inet static
address 192.168.0.112
netmask 255.255.255.0
network 192.168.0.0
broadcast 192.168.0.255
# gateway 192.168.0.1
dns-nameservers 213.75.63.36 213.75.63.70 192.168.0.1
# dns-nameservers 192.168.0.1 192.168.0.2
# dns-search mynet.int internal.mynet.int ops.mynet.int
auto eth1
iface eth1 inet static
address 188.204.140.195
netmask 255.255.255.224
network 188.204.140.192
broadcast 188.204.140.223
gateway 188.204.140.193
dns-nameservers 213.75.63.36 213.75.63.70 192.168.0.1
NB for dns-search or dns-domain to work in the above stanzas you need to have the resolvconf package installed. Otherwise you will have to add the following to /etc/resolve.conf
search mynet.int search internal.mynet.int ...
SNMP
run snmpconf
/etc/default/snmpd get rid of 127.0.0.1
/etc/snmp/snmpd.conf should have at least the following:
rocommunity public default -V systemonly rocommunity euhostro
to define the community and
agentAddress agentaddress
in order to allow the daemon to listen to all IP adresses instead of only localhost
test with
snmpwalk -v1 -c euhostro localhost
then from another server:
snmpwalk -v1 -c euhostro IPADRESS
sshd chroot for sftp
vi /etc/ssh/sshd_config
#Subsystem sftp /usr/libexec/openssh/sftp-server Subsystem sftp internal-sftp
at the end
Match Group sftp
ChrootDirectory /var/www/www.sitename.ext/
ForceCommand internal-sftp
AllowTcpForwarding no
or per user
Match User myguy
ChrootDirectory /var/www/www.sitename.ext/
ForceCommand internal-sftp
AllowTcpForwarding no
systemctl restart sshd
Note: make sure that the ChrootDirectory itself is owned by root!
Proftpd (depreciated)
/etc/proftpd/proftpd.conf add
DefaultRoot ~/../../
Also add /bin/false to /etc/shells
This allows users to log in with ftp, but not with ssh
For AWStats
LogFormat awstats "%t %h %u %m %f %s %b" ExtendedLog /var/log/xferlog read,write awstats TransferLog none RequireValidShell off
Hosting scripts and directories
mkdir /home/adm_usr/webserveradmin/ -p mkdir /opt/myhost/ -p mkdir /opt/weblog/etc -p mkdir /opt/weblog/src -p
copy the stuff from another webserver into these dirs and find and replace the servername in these directories.
mkdir /home/sites/servername.xxx.com/site/sitestats/ -p mkdir /home/sites/servername.xxx.com/site/sitestats/servername.xxx.com/ mkdir /home/sites/USGP.xxx.com/logs/ mkdir /home/sites/USGP.xxx.com/sites/ftpstats chown razor /home/sites/servername.xxx.com/site -R
APC
add
apc.shm_size=512M
to /etc/php5/apache2/conf.d/20-apc.ini
Apache2
log rotation
/etc/logrotate.d/apache2
/var/log/statistics {
daily
missingok
rotate 8
compress
}
/var/log/apache2/*.log {
prerotate
# Run the central statistics before rotating the logs
/opt/myhost/statisticsSERVERNAMEweb.sh
# Then we split the logs for the virtual hosts
/opt/myhost/apachelogsplit.sh
# Run the individual site stats
/opt/myhost/sitestatistics.sh
echo "All done for the day" >> /var/log/statistics
date >> /var/log/statistics
endscript
daily
missingok
rotate 7
compress
delaycompress
notifempty
create 640 root adm
sharedscripts
postrotate
if [ -f "`. /etc/apache2/envvars ; echo ${APACHE_PID_FILE:-/var/run/apache2.pid}`" ]; then
/etc/init.d/apache2 reload > /dev/null
fi
endscript
}
touch /var/log/statistics mkdir /var/log/apache2/virts mkdir /var/log/apache2/awstats
apache2 conf
/etc/apache2/apache2.conf change LogFormat and add %v to the beginning of the the combined format
LogFormat "%v %h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
And also check the values of
<IfModule mpm_prefork_module>
StartServers 100
MinSpareServers 80
MaxSpareServers 150
MaxClients 250
MaxRequestsPerChild 0
</IfModule>
and make sure
Options -Indexes
is in there!
/etc/apache2/ports.conf
NameVirtualHost *:80
Listen 80
<IfModule mod_ssl.c>
NameVirtualHost *:443
Listen 443
</IfModule>
<IfModule mod_gnutls.c>
Listen 443
</IfModule>
Creating a proxy passthrough
/etc/apache2/apache2.conf or /etc/apache2/mods-enabled/proxy.conf
# XXX change: This forces it to proxy to the monitor server # Requires libapache2-mod-proxy-html and a2enmod proxy ProxyRequests Off #ProxyPass / http://monitor.mynet.int/ #ProxyPassReverse / http://monitor.mynet.int/ ProxyPass / http://192.168.0.210/ ProxyPassReverse / http://192.168.0.210/
NB you can also do
ProxyPass /internal http://192.168.0.210/ ProxyPassReverse /internal http://192.168.0.210/
Which will make requests to http://external.domain/internal/foo go to http://192.168.0.210/foo. Note no trailing slashes!
Make sure all the proxy modules are enabled and restart the server after changes, don't reload.
/etc/apache2/sites-available/default
move this file to 000-default.conf to ensure it gets loaded first by apache
change
ServerName IPADDRESS DocumentRoot /home/sites/servername.xxx.com/site
and add
Redirect /stats http://servername.xxx.com/sitestats/mywraith.xxx.com/index.php
Redirect /livestats http://servername.xxx.com/cgi-bin/awstats.pl?config=mywraith
# AliasMatch ^/mailstats(.*) /home/sites/servername.xxx.com/mailstats/awstats.servername.mail.html
AliasMatch ^/ftpstats(.*) /home/sites/servername.xxx.com/ftpstats/awstats.servername.ftp.html
<Directory /home/sites/servername.xxx.com/>
Options Indexes FollowSymLinks MultiViews
AllowOverride Options Authconfig
Order allow,deny
allow from all
# This directive allows us to have apache2's default start page
# in /apache2-default/, but still have / go to the right place
</Directory>
to the bottom
So it should look something like:
<VirtualHost *:80>
ServerAdmin webmaster@localhost
DocumentRoot /home/sites/USGP.xxx.com/site
<Directory />
Options FollowSymLinks
AllowOverride None
</Directory>
<Directory /var/www/>
Options Indexes FollowSymLinks MultiViews
AllowOverride None
Order allow,deny
allow from all
</Directory>
ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/
<Directory "/usr/lib/cgi-bin">
AllowOverride None
Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch
Order allow,deny
Allow from all
</Directory>
ErrorLog /var/log/apache2/error.log
# Possible values include: debug, info, notice, warn, error, crit,
# alert, emerg.
LogLevel warn
CustomLog /var/log/apache2/access.log combined
Alias /doc/ "/usr/share/doc/"
<Directory "/usr/share/doc/">
Options Indexes MultiViews FollowSymLinks
AllowOverride None
Order deny,allow
Deny from all
Allow from 127.0.0.0/255.0.0.0 ::1/128
</Directory>
</VirtualHost>
make sure this links from /etc/apache2/sites-enabled/000-default as this becomes the fallback site for any IP or domain name not otherwise used.
Lets Encrypt
Because the Debian backport version is buggy, don't just run certbot --apache but you have to run the certbot for each domain (and serveraliases) you want it to work with as such: Note - nowadays you will need to do it this way!
certbot --authenticator webroot --installer apache -d www.domain.ext -w /var/www/www.domain.ext/site/ certbot -a webroot -i apache -d www.domain.ext -w /var/www/www.domain.ext/site/ certbot -a webroot -i apache -d www.domain.ext,domain.ext,other.domain.ext -w /var/www/www.domain.ext/site/
After adding / changing certificates make sure you reload the apache2 server.
If you see a warning with a lock and a yellow triangle from Firefox it means the page is serving mixed content (http + https). When inspecting the certificate itself it will look like there is no verifing agency for the certificate. This has nothing to do with the validity or verification of the certificate, it's purely there because of mixed content.
You can inspect the history of a certificate with the following URL
https://crt.sh/?q=%25linkielist.com
To renew certificates (put this in a cronjob)
certbot renew
To test
certbot renew --dry-run
Certbot documentation for apache + jessie
OLD
certbot -d www.domain.ext -d domain.ext -d www.alias.ext -d alias.ext --apache
/etc/apache2/sites-available/default-ssl
generate a certificate using certbot for the servername
certbot certonly edgarinet.edgarbv.com
them move the file to 000-default-ssl.conf to ensure it gets loaded first
<IfModule mod_ssl.c>
<VirtualHost *:443>
Alias /roundcube /var/lib/roundcube
ServerAdmin webmaster@localhost
DocumentRoot /var/www
<Directory />
Options FollowSymLinks
AllowOverride None
</Directory>
<Directory /var/www/>
Options FollowSymLinks MultiViews
AllowOverride None
Order allow,deny
allow from all
</Directory>
ErrorLog ${APACHE_LOG_DIR}/error.log
LogLevel warn
CustomLog ${APACHE_LOG_DIR}/ssl_access.log combined
<FilesMatch "\.(cgi|shtml|phtml|php)$">
SSLOptions +StdEnvVars
</FilesMatch>
<Directory /usr/lib/cgi-bin>
SSLOptions +StdEnvVars
</Directory>
BrowserMatch "MSIE [2-6]" \
nokeepalive ssl-unclean-shutdown \
downgrade-1.0 force-response-1.0
SSLCertificateFile /etc/letsencrypt/live/edgarinet.edgarbv.com/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/edgarinet.edgarbv.com/privkey.pem
Include /etc/letsencrypt/options-ssl-apache.conf
</VirtualHost>
</IfModule>
depreciated /etc/apache2/sites-available/default-ssl
Don't do this because LetsEncrypt!
Because we edited ports conf, we need to change:
<VirtualHost *:443>
and of course very important are
SSLEngine on
SSLCertificateFile /etc/ssl/certs/ssl-cert-snakeoil.pem
SSLCertificateKeyFile /etc/ssl/private/ssl-cert-snakeoil.key
SSLCACertificatePath /etc/ssl/certs/
default-ssl should look something like
<IfModule mod_ssl.c>
<VirtualHost *:443>
ServerAdmin webmaster@localhost
DocumentRoot /var/www
<Directory />
Options FollowSymLinks
AllowOverride None
</Directory>
<Directory /var/www/>
Options FollowSymLinks MultiViews
AllowOverride None
Order allow,deny
allow from all
</Directory>
ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/
<Directory "/usr/lib/cgi-bin">
AllowOverride None
Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch
Order allow,deny
Allow from all
</Directory>
ErrorLog ${APACHE_LOG_DIR}/error.log
LogLevel warn
CustomLog ${APACHE_LOG_DIR}/ssl_access.log combined
SSLEngine on
SSLCertificateFile /etc/ssl/certs/ssl-cert-snakeoil.pem
SSLCertificateKeyFile /etc/ssl/private/ssl-cert-snakeoil.key
SSLCACertificatePath /etc/ssl/certs/
<FilesMatch "\.(cgi|shtml|phtml|php)$">
SSLOptions +StdEnvVars
</FilesMatch>
<Directory /usr/lib/cgi-bin>
SSLOptions +StdEnvVars
</Directory>
BrowserMatch "MSIE [2-6]" \
nokeepalive ssl-unclean-shutdown \
downgrade-1.0 force-response-1.0
BrowserMatch "MSIE [17-9]" ssl-unclean-shutdown
</VirtualHost>
</IfModule>
make sure that this links from /etc/apache2/sites-enabled/000-default-ssl as we need this to be the first SNI site.
Public Key Pinning Extension for HTTP (HPKP)
Disable SSLv3
/etc/apache2/mods-available/ssl.conf Make sure it has SSLv3 disabled, check the line:
SSLProtocol All -SSLv2 -SSLv3
Standard site
Should look something like:
<VirtualHost *:80>
ServerName robin.xxx.com
DocumentRoot /home/sites/robin.xxx.com/site
ServerAdmin red@email.com
ServerAlias xxx.com
Redirect /stats http://USGP.xxx.com/sitestats/robin.xxx.com/index.php
Redirect /livestats http://robin.xxx.com/cgi-bin/awstats.pl?config=robin.xxx.com
<Directory /home/sites/robin.xxx.com/site/>
Options FollowSymLinks
AllowOverride All
Order allow,deny
allow from all
</Directory>
ErrorLog /var/log/apache2/error.log
LogLevel warn
CustomLog /var/log/apache2/access.log combined
ServerSignature On
</VirtualHost>
ensure sites have indexes
cp /var/www/index.html /home/sites cp /var/www/index.html /home/sites/servername.xxx.com/site/
listvirts
/etc/apache2/listvirts (NB has to start at group 100!)
# nb make sure first site after the original starts at 100! mywraith.xxx.com - site0 some.site.com - site100
OLD
create /etc/apache2/sites-available/82.95.91.75 with DocumentRoot /home/sites by hand!
/OLD
create /etc/apache2/sites-available/servername.xxx.com
link it in in sites-enabled
check both sites to see if they go to different indexes.
a2enmod rewrite (or a2enmod for options list)
vi /etc/apache2/conf.d/awstats
Alias /awstatsicon/ /usr/share/awstats/icon/
Post configuration
AWStats
cp /usr/share/doc/awstats/examples/apache.conf /etc/apache2/conf.d/awstats
Make sure that /var/log/apache2 is readable by www-data
chgrp /var/log/apache2/ www-data -R
touch /var/log/statistics
/etc/awstats/model.conf
tar xzvf /usr/share/doc/awstats/examples/awstats.model.conf.gz cp /usr/share/doc/awstats/examples/awstats.model.conf /etc/awstats/model.conf
Changes in the model.conf for our scripts:
LogFile="thislogfile" LogFormat="%virtualname %host %other %logname %time1 %methodurl %code %bytesd %refererquot %uaquot" SiteDomain="thissitedomain" HostAliases="localhost 127.0.0.1 REGEX[thisdomname\.(thisdomext)$]" DNSLookup=1 DirData="/var/log/apache2/awstats" DirIcons="/awstatsicon" AllowFullYearView=3 SaveDatabaseFilesWithPermissionsForEveryone=1 KeepBackupOfHistoricFiles=1 DebugMessages=1
cp /etc/awstats/model.conf /etc/awstats/awstats.servername.xxx.com.conf
Edit the following directives:
LogFile="/var/log/apache2/access.log" SiteDomain="servername.xxx.com" HostAliases="localhost 127.0.0.1 REGEX[servername.xxx\.(com|nl)$]"
Create the index.php file in /home/sites/servername.xxx.com/site/sitestats/servername.xxx.com/
<?
Header('Location: http://servername.xxx.com/sitestats/servername.xxx.com/awstats.zpress.xxx.com.html')
?>
Also do this for the serverIP
copy /etc/awstats/awstats.servername* (ftp / mail / web)
run the statisticsrun in /etc/logrotate.d/apache2 by hand to see how it all goes! ie.
cat /opt/myhost/statisticsSERVERNAME.sh
and run this line by line.
cp /opt/weblog/src/weblog_files/graphs/ /home/sites/USGP.xxx.com/site/webloggraphs/ -R
testing
When testing, it's sometimes useful to delete the following:
/var/cache/awstats/* -R <- generated static files dir
/var/lib/awstats/* <- database directory
in /etc/cron.d/awstats are the run commands to generate the files.
Firewall
See Installing a new mailserver
For instructions on
Postfix and Procmail, as well as Dovecot (for mail pickup), Amavis-new and ClamAV for antivirus and Roundcube webmail
NB don't forget to
postmap virtual postmap transport
spamassassin (knowledgebase page)
other stuff
change the mysql password
set up disk quotas (Quota Howto)
backup scripts in /etc/crontab
00 2 * * * root /opt/myhost/mysqldatasnapdaily.sh 00 3 * * 7 root /opt/myhost/mysqldatasnapweekly.sh 00 4 1 * * root /opt/myhost/mysqldatasnapmonthly.sh
and
mkdir /home/store mkdir /home/store/daily mkdir /home/store/weekly mkdir /home/store/monthly
Add to Cacti