Hi Michael,
this patch looks quite good I think. A few small comments below...
On Tue, 2017-08-15 at 22:43 +0200, Wolfgang Apolinarski wrote:
- Updated to apache 2.4
- Updated the htpasswd generation to use the more secure bcrypt
algorithm
config/httpd/httpd.conf | 4 +- config/httpd/loadmodule.conf | 10 +- config/httpd/server-tuning.conf | 10 +- config/httpd/ssl-global.conf | 5 - config/httpd/vhosts.d/ipfire-interface-ssl.conf | 20 +- config/httpd/vhosts.d/ipfire-interface.conf | 18 +- config/httpd/vhosts.d/nagios.conf | 14 +- config/httpd/vhosts.d/openmailadmin.conf | 3 +- config/icinga/icinga.conf | 6 - config/nagiosql/nagios.conf | 17 +- config/owncloud/owncloud.conf | 3 +- config/php/php.ini | 2 +- config/phpSANE/phpSANE.conf | 3 +- config/rootfiles/common/apache2 | 604 +++++++++++++++++------- config/rootfiles/common/php | 7 +- lfs/apache2 | 28 +- src/setup/passwords.c | 2 +- 17 files changed, 498 insertions(+), 258 deletions(-)
diff --git a/config/httpd/httpd.conf b/config/httpd/httpd.conf index 9c1fb2b10..14dcc735c 100644 --- a/config/httpd/httpd.conf +++ b/config/httpd/httpd.conf @@ -65,7 +65,6 @@ Include /etc/httpd/conf/global.conf
# associate MIME types with filename extensions TypesConfig /etc/mime.types -DefaultType text/plain
# global (server-wide) SSL configuration, that is not specific to # any virtual host @@ -80,8 +79,7 @@ Include /etc/httpd/conf/ssl-global.conf AccessFileName .htaccess # and never show them <Files ~ "^.ht">
- Order allow,deny
- Deny from all
- Require all denied
</Files>
# List of resources to look for when the client requests a directory diff --git a/config/httpd/loadmodule.conf b/config/httpd/loadmodule.conf index e30f79b28..249221e8c 100644 --- a/config/httpd/loadmodule.conf +++ b/config/httpd/loadmodule.conf @@ -1,8 +1,11 @@ LoadModule authn_file_module /usr/lib/apache/mod_authn_file.so +LoadModule unixd_module /usr/lib/apache/mod_unixd.so #LoadModule authn_dbm_module /usr/lib/apache/mod_authn_dbm.so #LoadModule authn_anon_module /usr/lib/apache/mod_authn_anon.so #LoadModule authn_dbd_module /usr/lib/apache/mod_authn_dbd.so #LoadModule authn_default_module /usr/lib/apache/mod_authn_default.so +LoadModule authn_core_module /usr/lib/apache/mod_authn_core.so +LoadModule authz_core_module /usr/lib/apache/mod_authz_core.so LoadModule authz_host_module /usr/lib/apache/mod_authz_host.so #LoadModule authz_groupfile_module /usr/lib/apache/mod_authz_groupfile.so LoadModule authz_user_module /usr/lib/apache/mod_authz_user.so @@ -10,7 +13,7 @@ LoadModule authz_user_module /usr/lib/apache/mod_authz_user.so #LoadModule authz_owner_module /usr/lib/apache/mod_authz_owner.so #LoadModule authz_default_module /usr/lib/apache/mod_authz_default.so LoadModule auth_basic_module /usr/lib/apache/mod_auth_basic.so -LoadModule auth_digest_module /usr/lib/apache/mod_auth_digest.so +#LoadModule auth_digest_module /usr/lib/apache/mod_auth_digest.so
Any reason this is suddenly deactivated? I know we don't really use this anywhere but just asking...
Yes, I wrote a mail on the 18th of May that explained the modules and the changes (ok, that is long ago). Since you suggested that fewer modules result in a smaller attack surface, I disabled this module. In general, auth digest is not secure and not even more secure than basic auth: To cite the apache documentation: "Another problem is that the storage of the passwords on the server is insecure. The contents of a stolen htdigest file can be used directly for digest authentication."
Since almost no one is using auth digest and it gives no security advantage, I thought that disabling the module makes sense.
#LoadModule dbd_module /usr/lib/apache/mod_dbd.so #LoadModule dumpio_module /usr/lib/apache/mod_dumpio.so #LoadModule ext_filter_module /usr/lib/apache/mod_ext_filter.so @@ -33,10 +36,10 @@ LoadModule setenvif_module /usr/lib/apache/mod_setenvif.so LoadModule mime_module /usr/lib/apache/mod_mime.so #LoadModule dav_module /usr/lib/apache/mod_dav.so #LoadModule status_module /usr/lib/apache/mod_status.so -LoadModule autoindex_module /usr/lib/apache/mod_autoindex.so +#LoadModule autoindex_module /usr/lib/apache/mod_autoindex.so
Why is the autoindex module deactivated?
I deactivated all modules that are currently not in use. Actually, the autoindex may lead to potential privacy issues, especially since the web server may be used by several extensions. Although the web server should only be reachable from the inner network, a user could change that configuration. If autoindexing is necessary, the module can be activated quite easily manually.
Of course, I can activate it, if you think that this should be the default.
#LoadModule asis_module /usr/lib/apache/mod_asis.so #LoadModule info_module /usr/lib/apache/mod_info.so -LoadModule cgi_module /usr/lib/apache/mod_cgi.so +LoadModule cgid_module /usr/lib/apache/mod_cgid.so #LoadModule dav_fs_module /usr/lib/apache/mod_dav_fs.so #LoadModule vhost_alias_module /usr/lib/apache/mod_vhost_alias.so #LoadModule negotiation_module /usr/lib/apache/mod_negotiation.so @@ -47,5 +50,6 @@ LoadModule dir_module /usr/lib/apache/mod_dir.so #LoadModule userdir_module /usr/lib/apache/mod_userdir.so LoadModule alias_module /usr/lib/apache/mod_alias.so LoadModule rewrite_module /usr/lib/apache/mod_rewrite.so +LoadModule socache_shmcb_module /usr/lib/apache/mod_socache_shmcb.so
Do we need this module?
Yes, it is used by the SSLSessionCache.
LoadModule ssl_module /usr/lib/apache/mod_ssl.so LoadModule php5_module /usr/lib/apache/libphp5.so diff --git a/config/httpd/server-tuning.conf b/config/httpd/server- tuning.conf index 90410186d..5edfb990e 100644 --- a/config/httpd/server-tuning.conf +++ b/config/httpd/server-tuning.conf @@ -17,10 +17,12 @@ MaxKeepAliveRequests 100 # KeepAliveTimeout 15
-MinSpareServers 1 -MaxSpareServers 10 -StartServers 2 -MaxClients 256 +MinSpareThreads 1 +MaxSpareThreads 11 +StartServers 1 +MaxRequestWorkers 10 +MaxConnectionsPerChild 100 +ThreadsPerChild 10
For what reason did you change the number of started servers and clients. We recently adjusted this a little since there were problems with update accelerator in large networks.
Ok, I will try to come up with a configuration that is quite similar to the current configuration, then. During my tests, the web server was always very responsive, even with extensions like owncloud, but there are plenty of configurations and systems that I cannot test, so sticking as close as possible to the existing configuration might be a good idea.
(...)
@@ -1146,13 +1452,21 @@ etc/httpd/conf/vhosts.d/ipfire-interface.conf #srv/web/ipfire/manual/vhosts/name-based.html.ja.utf8 #srv/web/ipfire/manual/vhosts/name-based.html.ko.euc-kr #srv/web/ipfire/manual/vhosts/name-based.html.tr.utf8 -#usr/bin/apr-1-config -#usr/bin/apu-1-config +#usr/bin/ab +#usr/bin/apxs +#usr/bin/dbmmanage +#usr/bin/htdbm +#usr/bin/htdigest +usr/bin/htpasswd +#usr/bin/httxt2dbm +#usr/bin/logresolve #usr/include/apache
(...)
-#usr/lib/libaprutil-1.so.0.5.3 -#usr/lib/pkgconfig/apr-1.pc -#usr/lib/pkgconfig/apr-util-1.pc -#usr/sbin/ab +usr/lib/apache/mod_watchdog.so +usr/lib/apache/mod_xml2enc.so usr/sbin/apachectl -#usr/sbin/apxs #usr/sbin/checkgid -#usr/sbin/dbmmanage #usr/sbin/envvars #usr/sbin/envvars-std +usr/sbin/fcgistarter #usr/sbin/htcacheclean -#usr/sbin/htdbm -#usr/sbin/htdigest -usr/sbin/htpasswd usr/sbin/httpd -#usr/sbin/httxt2dbm -#usr/sbin/logresolve #usr/sbin/rotatelogs
All the commands that have been moved from /usr/sbin to /usr/bin must be deleted manually when installing the core update.
This would be htpasswd only, right? Can I do anything that eases the process?
+#usr/share/man/man1/ab.1
install : $(TARGET)
@@ -75,7 +78,7 @@ $(subst %,%_MD5,$(objects)) : $(TARGET) : $(patsubst %,$(DIR_DL)/%,$(objects)) @$(PREBUILD) @rm -rf $(DIR_APP) && cd $(DIR_SRC) && tar jxf $(DIR_DL)/$(DL_FILE)
- cd $(DIR_APP) && patch -Np1 -i $(DIR_DL)/httpd-2.2.2-config-
1.patch
cd $(DIR_APP) && patch -Np0 -i $(DIR_DL)/PR61382-Fix.patch
### Add IPFire's layout, too echo "# IPFire layout" >> $(DIR_APP)/config.layout
@@ -103,14 +106,15 @@ $(TARGET) : $(patsubst %,$(DIR_DL)/%,$(objects)) echo "</Layout>" >> $(DIR_APP)/config.layout
cd $(DIR_APP) && ./configure --enable-layout=IPFire \
--enable-ssl --enable-mods-shared=all --enable-proxy
--enable-ssl --enable-mods-shared=all --enable-proxy --with-mpm=event
And lastly the big question: Why MPM? What happened to prefork? Advantages/disadvantages?
Prefork is quite memory intensive. Event is stable and chosen as default on many distributions. The apache documentation states: "In the case of Unix, the decision as to which MPM is installed is based on two questions: 1. Does the system support threads? 2. Does the system support thread-safe polling (Specifically, the kqueue and epoll functions)? If the answer to both questions is 'yes', the default MPM is event. If The answer to #1 is 'yes', but the answer to #2 is 'no', the default will be worker. If the answer to both questions is 'no', then the default MPM will be prefork. In practical terms, this means that the default will almost always be event, as all modern operating systems support these two features."
Since we support the features and a lower memory footprint is in our interest, I would suggest that event MPM is the right choice here.
There are possible disadvantages that non-thread-safe modules could break. To the best of my knowledge, we do not use these kind of modules and they are actually quite old and usually unsupported.
I still remember that apache was the common example for explaining how forking network applications work. Eventually, multi-threading made it's way... (although preforking is still supported, of course!)
(...)
Best regards, Wolfgang
Hi,
sorry for the late reply.
On Fri, 2017-08-18 at 19:47 +0200, Wolfgang Apolinarski wrote:
Hi Michael,
this patch looks quite good I think. A few small comments below...
On Tue, 2017-08-15 at 22:43 +0200, Wolfgang Apolinarski wrote:
- Updated to apache 2.4
- Updated the htpasswd generation to use the more secure bcrypt
algorithm
config/httpd/httpd.conf | 4 +- config/httpd/loadmodule.conf | 10 +- config/httpd/server-tuning.conf | 10 +- config/httpd/ssl-global.conf | 5 - config/httpd/vhosts.d/ipfire-interface-ssl.conf | 20 +- config/httpd/vhosts.d/ipfire-interface.conf | 18 +- config/httpd/vhosts.d/nagios.conf | 14 +- config/httpd/vhosts.d/openmailadmin.conf | 3 +- config/icinga/icinga.conf | 6 - config/nagiosql/nagios.conf | 17 +- config/owncloud/owncloud.conf | 3 +- config/php/php.ini | 2 +- config/phpSANE/phpSANE.conf | 3 +- config/rootfiles/common/apache2 | 604 +++++++++++++++++------- config/rootfiles/common/php | 7 +- lfs/apache2 | 28 +- src/setup/passwords.c | 2 +- 17 files changed, 498 insertions(+), 258 deletions(-)
diff --git a/config/httpd/httpd.conf b/config/httpd/httpd.conf index 9c1fb2b10..14dcc735c 100644 --- a/config/httpd/httpd.conf +++ b/config/httpd/httpd.conf @@ -65,7 +65,6 @@ Include /etc/httpd/conf/global.conf
# associate MIME types with filename extensions TypesConfig /etc/mime.types -DefaultType text/plain
# global (server-wide) SSL configuration, that is not specific to # any virtual host @@ -80,8 +79,7 @@ Include /etc/httpd/conf/ssl-global.conf AccessFileName .htaccess # and never show them <Files ~ "^.ht">
- Order allow,deny
- Deny from all
- Require all denied
</Files>
# List of resources to look for when the client requests a directory diff --git a/config/httpd/loadmodule.conf b/config/httpd/loadmodule.conf index e30f79b28..249221e8c 100644 --- a/config/httpd/loadmodule.conf +++ b/config/httpd/loadmodule.conf @@ -1,8 +1,11 @@ LoadModule authn_file_module /usr/lib/apache/mod_authn_file.so +LoadModule unixd_module /usr/lib/apache/mod_unixd.so #LoadModule authn_dbm_module /usr/lib/apache/mod_authn_dbm.so #LoadModule authn_anon_module /usr/lib/apache/mod_authn_anon.so #LoadModule authn_dbd_module /usr/lib/apache/mod_authn_dbd.so #LoadModule authn_default_module /usr/lib/apache/mod_authn_default.so +LoadModule authn_core_module /usr/lib/apache/mod_authn_core.so +LoadModule authz_core_module /usr/lib/apache/mod_authz_core.so LoadModule authz_host_module /usr/lib/apache/mod_authz_host.so #LoadModule authz_groupfile_module /usr/lib/apache/mod_authz_groupfile.so LoadModule authz_user_module /usr/lib/apache/mod_authz_user.so @@ -10,7 +13,7 @@ LoadModule authz_user_module /usr/lib/apache/mod_authz_user.so #LoadModule authz_owner_module /usr/lib/apache/mod_authz_owner.so #LoadModule authz_default_module /usr/lib/apache/mod_authz_default.so LoadModule auth_basic_module /usr/lib/apache/mod_auth_basic.so -LoadModule auth_digest_module /usr/lib/apache/mod_auth_digest.so +#LoadModule auth_digest_module /usr/lib/apache/mod_auth_digest.so
Any reason this is suddenly deactivated? I know we don't really use this anywhere but just asking...
Yes, I wrote a mail on the 18th of May that explained the modules and the changes (ok, that is long ago). Since you suggested that fewer modules result in a smaller attack surface, I disabled this module. In general, auth digest is not secure and not even more secure than basic auth: To cite the apache documentation: "Another problem is that the storage of the passwords on the server is insecure. The contents of a stolen htdigest file can be used directly for digest authentication."
Since almost no one is using auth digest and it gives no security advantage, I thought that disabling the module makes sense.
Very good reason and explanation. I just wanted this to be documented here in case someone is wondering later.
#LoadModule dbd_module /usr/lib/apache/mod_dbd.so #LoadModule dumpio_module /usr/lib/apache/mod_dumpio.so #LoadModule ext_filter_module /usr/lib/apache/mod_ext_filter.so @@ -33,10 +36,10 @@ LoadModule setenvif_module /usr/lib/apache/mod_setenvif.so LoadModule mime_module /usr/lib/apache/mod_mime.so #LoadModule dav_module /usr/lib/apache/mod_dav.so #LoadModule status_module /usr/lib/apache/mod_status.so -LoadModule autoindex_module /usr/lib/apache/mod_autoindex.so +#LoadModule autoindex_module /usr/lib/apache/mod_autoindex.so
Why is the autoindex module deactivated?
I deactivated all modules that are currently not in use. Actually, the autoindex may lead to potential privacy issues, especially since the web server may be used by several extensions. Although the web server should only be reachable from the inner network, a user could change that configuration. If autoindexing is necessary, the module can be activated quite easily manually.
Of course, I can activate it, if you think that this should be the default.
No this makes sense and is fine.
We don't use the listing anywhere.
#LoadModule asis_module /usr/lib/apache/mod_asis.so #LoadModule info_module /usr/lib/apache/mod_info.so -LoadModule cgi_module /usr/lib/apache/mod_cgi.so +LoadModule cgid_module /usr/lib/apache/mod_cgid.so #LoadModule dav_fs_module /usr/lib/apache/mod_dav_fs.so #LoadModule vhost_alias_module /usr/lib/apache/mod_vhost_alias.so #LoadModule negotiation_module /usr/lib/apache/mod_negotiation.so @@ -47,5 +50,6 @@ LoadModule dir_module /usr/lib/apache/mod_dir.so #LoadModule userdir_module /usr/lib/apache/mod_userdir.so LoadModule alias_module /usr/lib/apache/mod_alias.so LoadModule rewrite_module /usr/lib/apache/mod_rewrite.so +LoadModule socache_shmcb_module /usr/lib/apache/mod_socache_shmcb.so
Do we need this module?
Yes, it is used by the SSLSessionCache.
Okay.
LoadModule ssl_module /usr/lib/apache/mod_ssl.so LoadModule php5_module /usr/lib/apache/libphp5.so diff --git a/config/httpd/server-tuning.conf b/config/httpd/server- tuning.conf index 90410186d..5edfb990e 100644 --- a/config/httpd/server-tuning.conf +++ b/config/httpd/server-tuning.conf @@ -17,10 +17,12 @@ MaxKeepAliveRequests 100 # KeepAliveTimeout 15
-MinSpareServers 1 -MaxSpareServers 10 -StartServers 2 -MaxClients 256 +MinSpareThreads 1 +MaxSpareThreads 11 +StartServers 1 +MaxRequestWorkers 10 +MaxConnectionsPerChild 100 +ThreadsPerChild 10
For what reason did you change the number of started servers and clients. We recently adjusted this a little since there were problems with update accelerator in large networks.
Ok, I will try to come up with a configuration that is quite similar to the current configuration, then. During my tests, the web server was always very responsive, even with extensions like owncloud, but there are plenty of configurations and systems that I cannot test, so sticking as close as possible to the existing configuration might be a good idea.
Yes. If apache is busy on an IPFire system it is usually with a number of files being streamed from the update accelerator cache. Therefore we should allow many connections and storage will then be the bottleneck.
(...)
@@ -1146,13 +1452,21 @@ etc/httpd/conf/vhosts.d/ipfire-interface.conf #srv/web/ipfire/manual/vhosts/name-based.html.ja.utf8 #srv/web/ipfire/manual/vhosts/name-based.html.ko.euc-kr #srv/web/ipfire/manual/vhosts/name-based.html.tr.utf8 -#usr/bin/apr-1-config -#usr/bin/apu-1-config +#usr/bin/ab +#usr/bin/apxs +#usr/bin/dbmmanage +#usr/bin/htdbm +#usr/bin/htdigest +usr/bin/htpasswd +#usr/bin/httxt2dbm +#usr/bin/logresolve #usr/include/apache
(...)
-#usr/lib/libaprutil-1.so.0.5.3 -#usr/lib/pkgconfig/apr-1.pc -#usr/lib/pkgconfig/apr-util-1.pc -#usr/sbin/ab +usr/lib/apache/mod_watchdog.so +usr/lib/apache/mod_xml2enc.so usr/sbin/apachectl -#usr/sbin/apxs #usr/sbin/checkgid -#usr/sbin/dbmmanage #usr/sbin/envvars #usr/sbin/envvars-std +usr/sbin/fcgistarter #usr/sbin/htcacheclean -#usr/sbin/htdbm -#usr/sbin/htdigest -usr/sbin/htpasswd usr/sbin/httpd -#usr/sbin/httxt2dbm -#usr/sbin/logresolve #usr/sbin/rotatelogs
All the commands that have been moved from /usr/sbin to /usr/bin must be deleted manually when installing the core update.
This would be htpasswd only, right? Can I do anything that eases the process?
No not really. We just need to make sure that we delete the old files in the core update before we extract the new ones.
+#usr/share/man/man1/ab.1
install : $(TARGET)
@@ -75,7 +78,7 @@ $(subst %,%_MD5,$(objects)) : $(TARGET) : $(patsubst %,$(DIR_DL)/%,$(objects)) @$(PREBUILD) @rm -rf $(DIR_APP) && cd $(DIR_SRC) && tar jxf $(DIR_DL)/$(DL_FILE)
- cd $(DIR_APP) && patch -Np1 -i $(DIR_DL)/httpd-2.2.2-config-
1.patch
cd $(DIR_APP) && patch -Np0 -i $(DIR_DL)/PR61382-Fix.patch
### Add IPFire's layout, too echo "# IPFire layout" >> $(DIR_APP)/config.layout
@@ -103,14 +106,15 @@ $(TARGET) : $(patsubst %,$(DIR_DL)/%,$(objects)) echo "</Layout>" >> $(DIR_APP)/config.layout
cd $(DIR_APP) && ./configure --enable-layout=IPFire \
--enable-ssl --enable-mods-shared=all --enable-proxy
--enable-ssl --enable-mods-shared=all --enable-proxy --with-mpm=event
And lastly the big question: Why MPM? What happened to prefork? Advantages/disadvantages?
Prefork is quite memory intensive. Event is stable and chosen as default on many distributions. The apache documentation states: "In the case of Unix, the decision as to which MPM is installed is based on two questions:
- Does the system support threads?
- Does the system support thread-safe polling (Specifically, the kqueue and epoll functions)?
If the answer to both questions is 'yes', the default MPM is event. If The answer to #1 is 'yes', but the answer to #2 is 'no', the default will be worker. If the answer to both questions is 'no', then the default MPM will be prefork. In practical terms, this means that the default will almost always be event, as all modern operating systems support these two features."
Since we support the features and a lower memory footprint is in our interest, I would suggest that event MPM is the right choice here.
Indeed.
There are possible disadvantages that non-thread-safe modules could break. To the best of my knowledge, we do not use these kind of modules and they are actually quite old and usually unsupported.
We don't use any modules that are not coming with apache2. So I do not expect any problems here.
I still remember that apache was the common example for explaining how forking network applications work. Eventually, multi-threading made it's way... (although preforking is still supported, of course!)
No, these are all very good reasons and you made the right decisions that I would support.
The only reason against is that more changes in one feature will potentially raise more issues and prefork has been working well on IPFire for years. But with apache I am willing to take this risk because it should be very managable.
Good work!
-Michael
(...)
Best regards, Wolfgang
-
Michael Tremer -
Wolfgang Apolinarski