[Discuss] systemd race condition problems

[adler at stephenadler.com (Stephen Adler)]

On 08/30/2014 08:27 AM, Stephen Adler wrote:
> Hi All,
>
> I'm having trouble with httpd and my subversion repository setup. I'm 
> using mod_dav_svn apache module to access my repository through httpd. 
> So here's the rub. I have everything configured and when I go to 
> update a file, I get an error that a particular file is not readable. 
> (permission denied)
>
> [Sat Aug 30 08:05:27.432294 2014] [:error] [pid 2063] (20014)Internal 
> error: [client 71.163.254.54:49330] Can't open file 
> '/export/telacode/base/SVNRepositories/TelaCodeRepository/format': 
> Permission denied
>
>
> The odd bit is the file is world readable.
>
> [root at telacode ~]# ls -l 
> /export/telacode/base/SVNRepositories/TelaCodeRepository/format
> -r--r--r--. 1 softadmin telacoderepository 2 Feb 12  2007 
> /export/telacode/base/SVNRepositories/TelaCodeRepository/format
>
>
> The solution to my problem is to restart apache with the command 
> 'apachectl restart'. Then I can update my repository through the http 
> mod_dav_svn.
>
> The file system is located on a raid 5 array
>
> [root at telacode ~]# df /export
> Filesystem      1K-blocks      Used  Available Use% Mounted on
> /dev/md3       1892209456 129889412 1664627680   8% /export
>
> and I'm thinking that the process which mounts/activates the file 
> system is slower than the apache service startup process. (rhel 7 now 
> uses systemd instead of sysVinit).
>
> So, any ideas on how to delay apache startup so that it is guaranteed 
> to start after the /export file system is fully mounted and available 
> to the system?
>
> Thanks. Steve.
> _______________________________________________
> Discuss mailing list
> Discuss at blu.org
> http://lists.blu.org/mailman/listinfo/discuss
>

Answering my own question.... I learned about 'systemd-analyze plot'. 
 From this nice graphical display of the startup process, I realized 
that my ypbind.service was taking a bit of time. (I'm working on 
migrating to LDAP at some later point...) Since there are a lot of 
permission issues set through ypbind, I thought it best to delay staring 
up apache until after the ypbind service was working. Thus all it took 
was to add ypbind.service to /usr/lib/systemd/system/httpd.service file 
in the 'After' tag list.

-------- cat /usr/lib/systemd/system/httpd.service  ------

[Unit]
Description=The Apache HTTP Server
After=network.target remote-fs.target nss-lookup.target ypbind.service

[Service]
Type=notify
EnvironmentFile=/etc/sysconfig/httpd
ExecStart=/usr/sbin/httpd $OPTIONS -DFOREGROUND
ExecReload=/usr/sbin/httpd $OPTIONS -k graceful
ExecStop=/bin/kill -WINCH ${MAINPID}
# We want systemd to give httpd some time to finish gracefully, but 
still want
# it to kill httpd after TimeoutStopSec if something went wrong during the
# graceful stop. Normally, Systemd sends SIGTERM signal right after the
# ExecStop, which would kill httpd. We are sending useless SIGCONT here 
to give
# httpd time to finish.
KillSignal=SIGCONT
PrivateTmp=true

[Install]
WantedBy=multi-user.target

-----------------------------

and now after a fresh reboot, my subversion repository access through 
httpd seems to be working again.

Cheers. Steve.