how does systemd determine service is stopped?

Our application uses an init.d script to start and stop the application as a service. Under CentOS 7, /sbin/init is symlinked to systemd, so I can start my application using either:

service myapp start

systemctl start myapp

The issue I am having is that running stop on the service using either service or systemctl will not stop my application. The output of systemctl status:

[<a href="https://getridbug.com/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="75071a1a01351b10164541">[email protected]</a> ~]# systemctl status myapp
myapp.service - SYSV: Service script to start/stop my application
   Loaded: loaded (/etc/rc.d/init.d/myapp)
   Active: inactive (dead) since Mon 2015-10-05 15:17:41 CEST; 22h ago
  Process: 31850 ExecStop=/etc/rc.d/init.d/myapp stop (code=exited, status=0/SUCCESS)
  Process: 21054 ExecStart=/etc/rc.d/init.d/myapp start (code=exited, status=0/SUCCESS)

Using the service command:

[<a href="https://getridbug.com/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="8efce1e1facee0ebedbeba">[email protected]</a> ~]# service myapp status
Local database at :3307 is started
Watchdog is running
Application is running

Why does systemctl think my application is not running? Could it be that systemctl is not calling the stop function because it thinks my application is already stopped?

Contents hide

Answers:

Method 1

Method 2

Answers:

Thank you for visiting the Q&A section on Magenaut. Please note that all the answers may not help you solve the issue immediately. So please treat them as advisements. If you found the post helpful (or not), leave a comment & I’ll get back to you as soon as possible.

Method 1

I don’t know about the service output, never having really used that tool, but according to systemctl the app is indeed stopped. That’s what the Active: inactive (dead) line means. (Loaded just means that systemd has the unit file loaded into memory; this should be true all the time, whether the application is running or not.)

If there is still actually a process running for the application, then that implies that the stop function isn’t working properly. However, this shouldn’t happen; once systemctl stops something, it will (after some timeout) forcibly kill all the processes started by that application, using cgroups. So unless your process runs as root and is intentionally breaking out of its cgroup (an extremely pathological behavior), systemd should have killed it. I’m unsure as to how this interacts with sysVinit emulation, though.

This whole issue is complicated by the fact that systemd is using its sysVinit emulation, using the init script as a unit file. It may be better to rewrite as a proper unit file, as described e.g. here. I would tend to recommend using Type=simple rather than Type=dbus, since it’s much more common for a program to have a --no-daemon switch or equivalent than for it to talk to dbus, and this is also easier to add to code you have control over.

Method 2

Why does systemctl think my application is not running?

Because, as Tom Hunt says, it isn’t running.

Could it be that systemctl is not calling the stop function because it thinks my application is already stopped?

No. It very clearly did call the stop function, and ran it as process #31850.

There are two possibilities here, neither of which are systemd problems:

At some point, you started your service programs directly, not as a systemd service. That’s what’s still running. Of course systemd won’t know about it.
The status functionality of your init.d script is faulty. It wouldn’t be the first such faulty init.d script in the history of the world.

myapp.service – SYSV: Service script to start/stop my application

That “SYSV:” there is a giveaway that your init.d script is poor. It doesn’t even have the LSB header block.

As Tom Hunt says, write some service units. Or remember the first rule for migration to systemd and just go and pinch the ones that have already been written. By the looks of it, you actually have three interdependent but distinct services, and should be writing multiple service units with those interdependencies expressed. If one of them is a database server listening on port 3307, then the first rule almost certainly applies.

Answers:

Method 1

Method 2

Further reading