Debug systemd service failures on Ubuntu VMs without restarting services blindly. Get a structured audit of failed units, journal errors, and port listeners to pinpoint issues fast.
This guide exists as a tested, deployable recipe in the Massed Compute MCP. Connect an AI agent to provision VMs, run audits, and get structured reports automatically. Learn more about MCP automation.
When your Ubuntu VM is running but services are failing, ports aren’t responding, or deployments break silently, you need visibility into what’s actually happening. This guide provides a systematic approach to audit systemd units, journal logs, and service states without restarting anything.
Use this when systemd units fail, services exit unexpectedly, or expected ports go silent after a deployment. The outcome is a structured audit that correlates failed units with journal errors and listening ports.
| Component | Purpose | Version |
|---|---|---|
| Ubuntu Server | Host OS | 24.04 LTS |
| systemd | Service management | 255+ |
| journalctl | Log inspection | Built-in |
| SSH | Remote access | OpenSSH 9.0+ |
| Resource | Minimum | Recommended |
|---|---|---|
| vCPU | 2 cores | 2+ cores |
| RAM | 4 GB | 8 GB |
| Storage | 20 GB | 40 GB |
| Network | SSH access | Public IP |
Step-by-Step Deployment
Verify VM Access
Confirm your Ubuntu VM is running and accessible via SSH. You’ll need the instance details including public IP, SSH key, and connection port.
# Test SSH connection
ssh -i /path/to/your/private-key -p 22 ubuntu@YOUR_VM_IP 'echo "Connection OK"'
# Should return: Connection OK
If connection fails, verify the VM is in “running” state and the security group allows SSH on port 22.
Check Systemd Unit Status
Get an overview of all systemd units and identify any in failed states. This gives you the big picture before diving into specific services.
# List all failed units
sudo systemctl --failed
# Get system status overview
sudo systemctl status
# Count failed units
sudo systemctl --failed --no-legend | wc -l
Note any units showing as “failed” or “error” – these will be your primary investigation targets.
Audit Failed Unit Logs
For each failed unit, examine the journal logs to understand why it failed. Look for error messages, missing dependencies, or configuration issues.
# Check logs for a specific failed unit
sudo journalctl -u SERVICE_NAME -n 25 --no-pager
# Get error-priority logs from the last hour
sudo journalctl -p err -S "1 hour ago" --no-pager
# Show critical and alert messages
sudo journalctl -p crit..alert --no-pager
Focus on recent entries and look for patterns like permission errors, network issues, or missing configuration files.
Check Service Dependencies
Understand which services depend on the failed units and might be affected by their failure.
# Show what depends on a service
sudo systemctl list-dependencies SERVICE_NAME
# Show what a service depends on
sudo systemctl list-dependencies SERVICE_NAME --reverse
# Check if dependencies are running
sudo systemctl is-active SERVICE_NAME
This helps you understand the cascade effects of failed services and prioritize which ones to fix first.
Audit Network Listeners
Compare expected listening ports with actual network listeners to identify services that should be running but aren’t responding.
# Show all listening ports
sudo netstat -tlnp
# Alternative using ss
sudo ss -tlnp
# Check specific port
sudo netstat -tlnp | grep :80
sudo netstat -tlnp | grep :443
Correlate missing listeners with failed services. If a web server should be on port 80 but isn’t listed, that confirms the service audit findings.
Generate Audit Report
Compile your findings into a structured report that documents failed units, error patterns, and missing listeners.
# Create audit summary
echo "=== System Audit $(date) ===" > /tmp/audit-report.txt
echo "Failed Units:" >> /tmp/audit-report.txt
sudo systemctl --failed --no-legend >> /tmp/audit-report.txt
echo "" >> /tmp/audit-report.txt
echo "Recent Errors:" >> /tmp/audit-report.txt
sudo journalctl -p err -S "1 hour ago" --no-pager -o short >> /tmp/audit-report.txt
# View the report
cat /tmp/audit-report.txt
This report gives you a clear picture of system health and specific issues to address.
Troubleshooting
SSH Connection Issues
If you can’t connect to the VM:
- Verify the VM is in “running” state via your cloud console
- Check security groups allow SSH (port 22) from your IP
- Remove old host keys:
ssh-keygen -R YOUR_VM_IP - Verify SSH key permissions:
chmod 600 /path/to/private-key
Permission Denied Errors
If you get permission errors when running commands:
- Ensure you’re using
sudofor system commands - Verify your user is in the sudo group:
groups $USER - Check if the journal is readable:
ls -la /var/log/journal
Too Many Failed Units
If you have many failed units:
- Focus on services critical to your application first
- Look for common error patterns across multiple units
- Check if a single dependency failure is causing cascading failures
- Use
systemctl list-dependencies --allto understand relationships
Skip All of This: Deploy with an AI Agent
This guide exists as a tested, machine-readable recipe in the Massed Compute MCP. Instead of running commands manually, connect an AI agent to audit your VM automatically.
{
"mcpServers": {
"massed-compute": {
"type": "http",
"url": "https://vm.massedcompute.com/api/mcp",
"headers": { "Authorization": "Bearer MC_TOKEN" }
}
}
}
Then say:
The agent matches your request against the recipe catalog, connects to your VM, runs the audit commands above, and returns a structured report with failed units, journal excerpts, and listener correlation. It stops on any SSH or permission issues rather than proceeding with incomplete data. Recipe tested June 10, 2026.
Quick Setup Guide
For teams that prefer a checklist approach:
- Connect: SSH to your Ubuntu VM
- Check:
sudo systemctl --failedfor failed units - Investigate:
sudo journalctl -u SERVICE_NAME -n 25for each failed service - Correlate:
sudo netstat -tlnpto match listeners with services - Document: Save failed unit names and key error messages
- Plan: Prioritize fixes based on service dependencies
FAQ
01 Why not just restart failed services immediately?
Restarting without understanding the root cause often leads to repeated failures. This audit approach helps you identify configuration issues, dependency problems, or resource constraints that need to be fixed before the service can run successfully.
02 What if I have too many failed units to check individually?
Focus on services critical to your application first. Use systemctl list-dependencies to understand which failures might be causing cascade effects. Often, fixing one key service resolves multiple dependent unit failures.
03 Can I run this audit on production systems safely?
Yes, all commands in this guide are read-only and won’t modify system state. They only query systemd status and journal logs. The approach specifically avoids restart commands to prevent production disruption.
04 How often should I run service audits?
Run audits after deployments, when monitoring alerts indicate issues, or as part of regular health checks. Consider automating the audit commands in monitoring scripts to catch service failures early.
05 What should I do with the audit results?
Use the audit to prioritize fixes: address critical service failures first, fix common error patterns, and update monitoring to catch similar issues faster. Document patterns in your runbooks for faster future diagnosis.











