Automatically acquire and renew certificates using mod_md and Automated Certificate Management Environment (ACME) in Identity Management (IdM)

Introduction

In a previous article, I demonstrated how to configure the Automatic Certificate Management Environment (ACME) feature included in the Identity Management (IdM) Dogtag Certificate Authority (CA). Specifically, I covered installation of IdM with random serial numbers, and how to enable the ACME service and expired certificate pruning. This article explains the management of ACME (currently a technology preview) with IdM and Red Hat Enterprise Linux (RHEL) clients.

Currently, mod_md is the only ACME client implementation completely supported and provided by Red Hat. For this article, I use Apache httpd with mod_md, designed to simplify and automate infrastructure security by reducing certificate lifetimes and avoiding manual actions that are prone to error. This article includes new features of mod_md in RHEL 9.5.

Overview and configuration of client mod_md

The mod_md module manages properties of domains for one or more Virtual Host and its main function is to supervise and renew certificates over the ACME protocol. The client implementation mod_md implements the http-01, tls-alpn-01, and dns-01 challenges (the last one is new in RHEL 9.5 implementation of mod_md). ACME challenges are validation methods needed to prove that the origin is legitimate. IdM understands both http-01 and dns-01. This challenge requires the client to provision an HTTP resource. The identifier is authorized when sufficient challenges (usually one per identifier) have been validated. Then the client finalizes the order, causing the CA to issue a new certificate for the client’s key pair (which it automatically generates).

As a starting point, assume a client in RHEL 9.5 is enrolled in IdM:

[client]# ipa host-show client.lab.example.com
 Host name: client.lab.example.com
 Platform: x86_64
 Operating system: 5.14.0-503.14.1.el9_5.x86_64
 Principal name: host/[email protected]
 Principal alias: host/[email protected]
 SSH public key fingerprint: SHA256:c7svndlUtfULyc5Pu+KPq/xBTLUHma/dVrWT4wh65K8 (ecdsa-sha2-nistp256), SHA256:qeuyhUxlW7N7oHgNYPa5VVHXN00lUUzTR/Su5hffYg0 (ssh-ed25519),
                          SHA256:dnyWFpEoS0gA7uPuErGi/29I5lMzbuU6Wn2ZuPVMzEQ (ssh-rsa)
 Password: False
 Keytab: True
 Managed by: client.lab.example.com

The first step is to install mod_md. As a dependency, the Apache httpd server package is automatically installed, as well. On the client enrolled in IdM:

[client]# dnf install mod_md

Next, enable the httpservice on the firewall. This is important not only to make the web server available, but also allows communication between ACME Dogtag CA in IdM and the mod_md client:

[client]# firewall-cmd --add-service http
success
[client]# firewall-cmd --runtime-to-permanent
Success

To allow mod_md to make outbound connections to the ACME server, you may need to adjust your SELinux policy. First check the current policy:

[client]# getsebool httpd_can_network_connect
httpd_can_network_connect --> off

If it has the value off, turn it to on:

[client]# setsebool -P httpd_can_network_connect on
[client]# getsebool httpd_can_network_connect
httpd_can_network_connect --> on

After that, enable Apache:

[client]# systemctl enable --now httpd

The following directory is created as a result of enabling and installing mod_md:

[client]# ls -ltra /var/lib/httpd/md/
total 4
drwx------. 3 apache apache  16 Nov 25 14:38 ..
drwxr-x---. 2 apache root  6 Nov 25 14:38 staging
drwxr-x---. 2 apache root  6 Nov 25 14:38 ocsp
-rw-------. 1 root   root   116 Nov 25 14:38 md_store.json
drwxr-x---. 2 apache root  6 Nov 25 14:38 challenges
drwxr-x---. 2 root   root  6 Nov 25 14:38 accounts
drwxr-xr-x. 6 root   root 88 Nov 25 14:38 .

In this directory, the key and certificate are saved as a running configuration. Every time the lifetime of the certificate has expired or the certificate has been revoked by the Dogtag CA for whatever reason, the certificate generated by IdM Dogtag CA is stored in the staging area available on the path /etc/httpd/state/md/staging/. After a graceful restart, the new certificate and key are moved from the staging area to the running configuration inside /var/lib/httpd/md/.

At this moment, we only have access to the plain site without certificates:

[client]# curl http://client.lab.example.com
Hello World!

Next, configure a Virtual Host in the client:

[client]# cat /etc/httpd/conf.d/acme.conf
LogLevel debug md:trace5
MDCertificateAuthority https://idm.lab.example.com/acme/directory
MDCertificateAgreement accepted
MDStapling on
MDomain client.lab.example.com
MDCheckInterval 30
MDRenewWindow 1%
MDStaplingRenewWindow 99%
MDRenewMode always
<VirtualHost *:443>
ServerName client.lab.example.com
SSLEngine on
</VirtualHost>
<Location "/server-status">
SetHandler server-status
</Location>

The parameters configured in the Virtual Host are:

• MDStapling enables the Online Certificate Status Protocol (OCSP) Stapling, which provokes your Apache server to check the status of certificates configured regularly. In this sample configuration, it’s set to on.
• MDCheckInterval is a value interpreted as a duration, and recognizes the following suffixes: d (days), h (hours), mi (minutes) and s (seconds). If the suffix is omitted, it defaults to seconds. The parameter corresponds to when the httpd refreshes the OCSP response, and when the httpd detects that a certificate is revoked or expired. It is the minimum time for httpdto notice any change in the certificate status, and as a consequence trigger any action to renew. The minimum is 1 second, but it is not recommended to use such a short time in production. For this example, it has a value of 30. This is a new option of mod_md in RHEL 9.5, and is still not documented in the directives of the module.
• MDRenewWindow is the percentage of the total lifetime for mod_md to get a new signed certificate once the certificate validity falls below the duration. In our example, it’s 1% of 90 days. That’s a little less than one day (0.9 days) before the expiration date, so about February 22. By default, certificates are valid for 90 days, and mod_md renews them, at the earliest, at 33% of the complete lifetime before expiry. So 30 days before it expires, mod_md gets a new signed certificate. When configuring this, consider that the time must be enough to perform all the operations regarding the renewal. For example, one minute would likely cause problems, because the old certificate would expire while the certificates are being renewed.
• MDStaplingRenewWindow is the proportion, or percentage, subtracted from the OCSP response’s validity period (12 hours) at which httpd will retrieve a fresh OCSP response. An absolute duration for the remaining validity can also be set (for example, 2h). The longer it’s not refreshed (a low percent), the better the performance. The shorter the interval (a large percentage), the more rapidly certificate revocations are spread to clients, so you must also consider service reliability. The OCSP may respond, but then the httpdmust notice a revocation as defined by MDCheckInterval. For example, if we set the MDStaplingRenewWindow to the minimum of 99% (which is about 6 minutes), but an MDCheckInterval of several hours, then the OCSP status will notice a certificate revocation, but httpdwon’t trigger any action to renew until the MDCheckIntervalis reached. The two values must be set proportionally, and according to your organization’s needs.

Now we have the client configured. Next, it’s time to trigger the generation of the key pair and certificate, and gather and store them in the client. For that, apply a restart on the httpd of the client while we are monitoring the file /var/log/httpd/error_log. In the error_log, we can observe the following:

[Mon Nov 25 14:50:25.433534 2024] [ssl:debug] [pid 2005:tid 2005] ssl_util_ssl.c(451): AH02412: [client.lab.example.com:443] Cert matches for name 'client.lab.example.com' [subject: CN=Apache Managed Domain Fallback / issuer: CN=Apache Managed Domain Fallback / serial: 9AF3A79DFD6D53A54DD7B7C803906240576AA95E / notbefore: Nov 25 13:50:25 2024 GMT / notafter: Dec  9 13:50:25 2024 GMT]
[Mon Nov 25 14:50:27.177938 2024] [md:notice] [pid 2008:tid 2011] AH10059: The Managed Domain client.lab.example.com has been setup and changes will be activated on next (graceful) server restart.
[Mon Nov 25 14:53:44.130279 2024] [md:trace1] [pid 2248:tid 2248] mod_md.c(1258): host 'client.lab.example.com' is covered by a Managed Domaina and is being provided with 1 key/certificate files.

As can be seen in the log, a graceful restart is required to apply the certificate to the site:

[client]# systemctl reload httpd
[client]# curl https://client.lab.example.com
Hello World!

In this process, the certificate and key are gathered from IdM CA and stored in httpd’s configuration directory:

[client]# ls -ltra /var/lib/httpd/md/domains/client.lab.example.com/
total 20
-rw-------. 1 root root 1704 Nov 25 14:53 privkey.pem
-rw-------. 1 root root 3242 Nov 25 14:53 pubcert.pem
-rw-------. 1 root root 5011 Nov 25 14:53 job.json
drwx------. 3 root root   36 Nov 25 14:53 ..
-rw-------. 1 root root  557 Nov 25 14:53 md.json
drwx------. 2 root root   75 Nov 25 14:53 .

Look in the IdM UI to verify that a new certificate was automatically issued by the Dogtag CA. In this example, it’s the certificate with Serial Number 11:

Operation and management of mod_md client with ACME certificates

For simplicity, I’m using sequential serial numbers in this article, but remember that for IdM to automatically prune expired old certificates, certificates must be generated with random serial numbers. This requires installing IdM with the random serial number (RSNv3) option, as seen in my previous post.

You can view information about the certificate using the curl command. The issuer is the IdM CA, and the validity period is about 3 months (the default for ACME server, as in the Let’s Encrypt profile):

[client]# curl -v https://client.lab.example.com 2>&1 | less
[...]
* Server certificate:
*  subject: CN=client.lab.example.com
*  start date: Nov 25 14:10:19 2024 GMT
*  expire date: Feb 23 14:10:19 2025 GMT
*  subjectAltName: host "client.lab.example.com" matched cert's "client.lab.example.com"
*  issuer: O=LAB.EXAMPLE.COM; CN=Certificate Authority
*  SSL certificate verify ok.

The ipa cert-show command returns the same information. Add the serial number of the certificate to limit the search:

[client]# ipa cert-show 11
[...]
Subject: CN=client.lab.example.com
 Subject DNS name: client.lab.example.com
 Issuer: CN=Certificate Authority,O=LAB.EXAMPLE.COM
 Not Before: Mon Nov 25 14:10:19 2024 UTC
 Not After: Sun Feb 23 14:10:19 2025 UTC
 Serial number: 11
 Serial number (hex): 0xC
 Revoked: False

In the URL of the Virtual Host, you can check the certificate and stapling information. With the parameters we provided in our Virtual Host, we have the following at the URL http://client.lab.example.com/server-status:

It’s important to note that the Managed Staplings responderipa-ca.lab.example.com must have a valid DNS record pointing to the IdM Dogtag CA server. As long as the certificate is valid, and until the expiry date, the httpdchecks whether a new certificate is needed (in an interval defined by MDCheckInterval). Until that happens, we see the following messages in /var/log/httpd/error_log (system time is GMT +1):

[Mon Nov 25 15:31:04.965083 2024] [md:debug] [pid 4741:tid 4744] mod_md_drive.c(117): 
AH10053: md(client.lab.example.com): no need to renew
[Mon Nov 25 15:31:04.965101 2024] [md:trace2] [pid 4741:tid 4744] md_reg.c(733): md[client.lab.example.com]: certificate(0) life[Mon, 25 Nov 2024 14:10:19 GMT - Sun, 
23 Feb 2025 14:10:19 GMT] warn[Fri, 14 Feb 2025 14:10:19 GMT - Sun, 23 Feb 2025 
14:10:19 GMT]
[Mon Nov 25 15:31:04.965113 2024] [md:debug] [pid 4741:tid 4744] mod_md_drive.c(237): 
AH10107: next run in 34 seconds

As you can see in the previous picture, the Renew property in the Managed Certificatessection is marked to occur 3 months from the moment the certificate is issued (November 25, which is 3 months minus 1 day to accomplish MDRenew Window 1%). It’s useful to change the profile of the certificates issued by the IdM Dogtag CA. By default, the expiration date is in 3 months (aligning with the Let’s Encrypt profile), so for test purposes we can change it to 20 minutes. To do that, dump the configuration file of the profile certificates of ACME in the IdM server. To alter the certificate profile lifetime on IdM Dogtag CA:

[idm]# ipa certprofile-show acmeIPAServerCert \
--out acmeIPAServerCert.cfg
------------------------------------------------------------
Profile configuration stored in file 'acmeIPAServerCert.cfg'
------------------------------------------------------------
 Profile ID: acmeIPAServerCert
 Profile description: ACME IPA service certificate profile
 Store issued certificates: False
Change policyset.serverCertSet.7.default.params.range to 20:
policyset.serverCertSet.7.default.params.range=20

And add this line:

policyset.serverCertSet.7.default.params.rangeUnit=minute

Finally, apply the change in profile:

[idm]# ipa certprofile-mod acmeIPAServerCert --file acmeIPAServerCert.cfg
 Profile ID: acmeIPAServerCert
 Profile description: ACME IPA service certificate profile
 Store issued certificates: False

Our profile lifetime for newly issued certificates by the Dogtag CA ACME sets the expiration time to 20 minutes after it is issued:

With this set, we revoke the most recently issued certificate on IdM CA. In this example, the certificate number is 25, but yours might be different:

Wait for OCSP to refresh the response (around 6 minutes, defined by the MDStaplingRenewWindow percentage). In the meantime, use the openssl command to check the status of the certificate by sending an OCSP request directly to the IdM CA’s OCSP responder. As a prerequisite, you must have the /etc/ipa/ca.crt from IdM copied to the client, and also port 8080/tcp enabled on the IdM firewall. With this accomplished, execute the following command (the path /tmp/ca.crton client contains the certificate of IdM /etc/ipa/ca.crt):

[client]# openssl ocsp -resp_text -CAfile /tmp/ca.crt -url http://ipa-ca.lab.example.com:8080/ca/ocsp -issuer /tmp/ca.crt -serial 25
OCSP Response Data:
[...]
  Issuer Name Hash: E68309DF0EDDE7133F81DE0A254606F8473B3BB7
  Issuer Key Hash: 0691C0F623FECAB34585E5286210C32567351FA4
  Serial Number: 19
Cert Status: revoked
Revocation Time: Nov 25 22:35:34 2024 GMT
This Update: Nov 25 22:41:59 2024 GMT
[...]
Response verify OK

25: revoked

This Update: Nov 25 22:41:59 2024 GMT

Revocation Time: Nov 25 22:35:34 2024 GMT

As you can see, the status is revoked, but mod_md still needs to overcome the time defined in the MDStaplingRenewWindow to refresh the cached OCSP response. This is also the first step for Apache to notice the revoked certificates. After the time defined has passed, you can see this in /var/log/httpd/error_log:

[Mon Nov 25 23:42:53.790883 2024] [md:debug] [pid 11013:tid 11015] 
md_result.c(254): md[client.lab.example.com] detail[The certificate for the 
managed domain has been renewed successfully and can be used (valid since Mon, 25 Nov 2024 22:42:53 GMT). A graceful server restart now is recommended.] 
[Mon Nov 25 23:42:53.790895 2024] [md:debug] [pid 11013:tid 11015] md_reg.c(1164): client.lab.example.com: staging done 
[Mon Nov 25 23:42:53.790948 2024] [md:trace3] [pid 11013:tid 11015] md_curl.c(633): cleanup curl instance 
[Mon Nov 25 23:42:53.792849 2024] [md:trace3] [pid 11013:tid 11015] md_store_fs.c(526): mk_group_dir /etc/httpd/state/md/staging perm set 
[Mon Nov 25 23:42:53.792903 2024] [md:trace3] [pid 11013:tid 11015] md_store_fs.c(526): mk_group_dir /etc/httpd/state/md/staging/client.lab.example.com perm set 
[Mon Nov 25 23:42:53.792912 2024] [md:trace2] [pid 11013:tid 11015] md_store_fs.c(643): storing in /etc/httpd/state/md/staging/client.lab.example.com/job.json 
[Mon Nov 25 23:42:53.793182 2024] [md:trace3] [pid 11013:tid 11015] mod_md.c(241): store event=0 on file /etc/httpd/state/md/staging/client.lab.example.com/job.json (group 4) 
[Mon Nov 25 23:42:53.793212 2024] [md:notice] [pid 11013:tid 11015] AH10059: The Managed Domain client.lab.example.com has been setup and changes will be activated on next (graceful) server restart.

The new certificate and key are stored on staging area:

[client]# ls -ltra /etc/httpd/state/md/staging/client.lab.example.com/
total 20
drwxr-xr-x. 3 apache root  36 Nov 25 23:42 ..
-rw-r--r--. 1 apache apache  557 Nov 25 23:42 md.json
-rw-r--r--. 1 apache apache 1886 Nov 25 23:42 privkey.pem
-rw-r--r--. 1 apache apache 3242 Nov 25 23:42 pubcert.pem
-rw-r--r--. 1 apache apache 4844 Nov 25 23:42 job.json
drwxr-xr-x. 2 apache apache   75 Nov 25 23:42 .

We can check the contents of the certificate, and see that it is different in the staging area than in the running configuration. We could also check the start date of the certificate (as you’ll see in the next example, when the certificate has expired):

[client]# md5sum /etc/httpd/state/md/staging/client.lab.example.com/pubcert.pem
9be4f94858a433b5e2c9ec4b8796725f  /etc/httpd/state/md/staging/client.lab.example.com/pubcert.pem

And now a reload of Apache moves the certificate and key from the staging area to the running configuration. In the running configuration:

[client]# md5sum /var/lib/httpd/md/domains/client.lab.example.com/pubcert.pem
78dd87869420e9c1c07d7f6fc3512d1d  /var/lib/httpd/md/domains/client.lab.example.com/pubcert.pem

After executing the reload, we can see that the certificate has been moved, and the staging area has been removed:

[client]# systemctl reload httpd
[client]# md5sum /var/lib/httpd/md/domains/client.lab.example.com/pubcert.pem
9be4f94858a433b5e2c9ec4b8796725f  /var/lib/httpd/md/domains/client.lab.example.com/pubcert.pem
[client]# ls -ltra /etc/httpd/state/md/staging/client.lab.example.com/
ls: cannot access '/etc/httpd/state/md/staging/client.lab.example.com/': No such file or directory

The expiration operation is more or less the same. As a starting point, we have the following certificate with serial number 122:

[client]# curl -v https://client.lab.example.com
[...]
* Server certificate:
*  subject: CN=client.lab.example.com
*  start date: Nov 28 10:27:25 2024 GMT
*  expire date: Nov 28 10:47:25 2024 GMT
*  subjectAltName: host "client.lab.example.com" matched cert's "client.lab.example.com"
*  issuer: O=LAB.EXAMPLE.COM; CN=Certificate Authority
*  SSL certificate verify ok.
[client]# ipa cert-show 122
[...]
  Subject: CN=client.lab.example.com
 Subject DNS name: client.lab.example.com
 Issuer: CN=Certificate Authority,O=LAB.EXAMPLE.COM
 Not Before: Thu Nov 28 10:27:25 2024 UTC
 Not After: Thu Nov 28 10:47:25 2024 UTC
 Serial number: 122
 Serial number (hex): 0x7A
 Revoked: False

After the lifetime has passed, and the certificate is expired, we can observe a similar entry in /var/log/httpd/error_log (system time is GMT+1):

[Thu Nov 28 11:47:27.719850 2024] [md:debug] [pid 19563:tid 19566] 
md_result.c(254): md[client.lab.example.com] detail[The certificate for the 
managed domain has been renewed successfully and can be used (valid since Thu,
 28 Nov 2024 10:47:27 GMT). A graceful server restart now is recommended.]
[Thu Nov 28 11:47:27.719853 2024] [md:debug] [pid 19563:tid 19566] 
md_reg.c(1164): client.lab.example.com: staging done

The corresponding key pair and certificate are saved in the staging area, and a reload copies them from staging to the running configuration, and removes them from the staging area:

[client]# ls -ltra /etc/httpd/state/md/staging/client.lab.example.com/
total 20
-rw-r--r--. 1 apache apache  557 Nov 28 11:47 md.json
-rw-r--r--. 1 apache apache 1886 Nov 28 11:47 privkey.pem
-rw-r--r--. 1 apache apache 3242 Nov 28 11:47 pubcert.pem
-rw-r--r--. 1 apache apache 5131 Nov 28 11:47 job.json

The certificate in staging area has the following serial number and start date:

[client]# openssl x509 -serial -startdate -noout -in /etc/httpd/state/md/staging/client.lab.example.com/pubcert.pem
serial=7B
notBefore=Nov 28 10:47:27 2024 GMT

The certificate in the running configuration (expired) has the following:

[client]# openssl x509 -serial -startdate -noout \
-in /var/lib/httpd/md/domains/client.lab.example.com/pubcert.pem
serial=7A
notBefore=Nov 28 10:27:25 2024 GMT

After a reload of the service, we can observe that the certificate on running configuration has been replaced by the certificate of the staging area:

[client]# systemctl reload httpd
[client]# ls -ltra /etc/httpd/state/md/staging/client.lab.example.com/
ls: cannot access '/etc/httpd/state/md/staging/client.lab.example.com/': No such file or directory
[client]# openssl x509 -serial -startdate -noout -in /var/lib/httpd/md/domains/client.lab.example.com/pubcert.pem
serial=7B
notBefore=Nov 28 10:47:27 2024 GMT

Using the curl command, verify that the server is using the new certificate:

[client]# curl -v https://client.lab.example.com 2>&1
[...]
* Server certificate:
*  subject: CN=client.lab.example.com
*  start date: Nov 28 10:47:27 2024 GMT
*  expire date: Nov 28 11:07:27 2024 GMT
*  subjectAltName: host "client.lab.example.com" matched cert's "client.lab.example.com"
*  issuer: O=LAB.EXAMPLE.COM; CN=Certificate Authority
*  SSL certificate verify ok.

Automating the process

To have a completely automated environment, it would be nice not to have to perform the reload operation of httpdmanually every time a new certificate is issued and needs to be moved from staging to the running configuration. This is where systemdcomes to the rescue.

With systemd, we can monitor whether a specific file exists, and when it does we can execute some actions on a service. First, create the corresponding .pathunit file to check for the existence of the certificate file in staging (remember that this path is usually empty because it only contains the newly renewed certificates before they’ve been configured on the Virtual Host):

[client]# cat /etc/systemd/system/acme.path
[Unit]
Description= Triggers the reload of httpd
[Path]
PathModified=/etc/httpd/state/md/staging/client.lab.example.com/pubcert.pem
[Install]
WantedBy=multi-user.target

Next, create a .service that corresponds to the acme.path (these must have the same name) that triggers the reload of httpd:

[client]# cat /etc/systemd/system/acme.service
[Unit]
Description=srv restarter
[Service]
Type=oneshot
ExecStart=/usr/bin/systemctl reload httpd.service

Afterwards, use daemon-reload and then enable the acme.path:

[client]# systemctl daemon-reload
[client]# systemctl enable --now acme.path

And now, every time that the pubcert.pem is detected in the staging area, a reload of the httpd service is triggered automatically. You don’t have to perform any action, and the complete lifecycle is automated. This monitoring covers both situations, when certificates are expired and also when certificates are revoked.

Multi-site certificate management

To manage multiple domains with the same Apache server, it’s useful to have a PathExistsGlobinstead of PathModified to make the environment scalable. This is useful when you manage more than one domain in your Virtual Host. For example, you could have the following domains in /etc/httpd/conf.d/acme.conf, each with its own MDomain:

[client]# cat /etc/httpd/conf.d/acme.conf
LogLevel debug md:trace5
MDCertificateAuthority https://idm.lab.example.com/acme/directory
MDCertificateAgreement accepted
MDStapling on
MDomain client.lab.example.com
MDCheckInterval 30
MDRenewWindow 1%
MDStaplingRenewWindow 99%
MDRenewMode always
<VirtualHost *:443>
ServerName client.lab.example.com
DocumentRoot /srv/www/
SSLEngine on
<Directory /srv/www>
  Require all granted
</Directory>
</VirtualHost>
MDomain www.lab.example.com
<VirtualHost *:443>
ServerName www.lab.example.com
DocumentRoot /srv/wwwv2/
SSLEngine on
<Directory /srv/wwwv2>
  Require all granted
</Directory>
</VirtualHost>
<Location "/server-status">
SetHandler server-status
</Location>

You must have the appropriate SELinux context for /srv/www/ and /srv/wwwv2/:

[client]# semanage fcontext -a -t httpd_sys_content_t "/srv/wwwv2(/.*)?"
[client]# semanage fcontext -a -t httpd_sys_content_t "/srv/wwwv(/.*)?"
[client]# restorecon -Rv /srv/www
[client]# restorecon -Rv /srv/wwwv2

Create a corresponding index.html file on both domains. After a restart of httpd, the sites are available:

[client]# curl https://client.lab.example.com
Hello World!
[client]# curl https://www.lab.example.com
Hello World v2!

They each have a unique certificate:

Each one also has its own running configuration and staging area:

[client]# ls -ltra /etc/httpd/state/md/staging/client.lab.example.com/
total 20
-rw-r--r--. 1 apache apache  557 Nov 28 16:17 md.json
-rw-r--r--. 1 apache apache 1886 Nov 28 16:17 privkey.pem
-rw-r--r--. 1 apache apache 3242 Nov 28 16:17 pubcert.pem
-rw-r--r--. 1 apache apache 5131 Nov 28 16:17 job.json
drwxr-xr-x. 2 apache apache   75 Nov 28 16:17 .
drwxr-xr-x. 4 apache root  63 Nov 28 16:20 ..
[client]# ls -ltra /etc/httpd/state/md/staging/www.lab.example.com/
total 20
drwxr-xr-x. 4 apache root  63 Nov 28 16:20 ..
-rw-r--r--. 1 apache apache  551 Nov 28 16:20 md.json
-rw-r--r--. 1 apache apache 1886 Nov 28 16:20 privkey.pem
-rw-r--r--. 1 apache apache 3233 Nov 28 16:20 pubcert.pem
-rw-r--r--. 1 apache apache 5089 Nov 28 16:20 job.json
drwxr-xr-x. 2 apache apache   75 Nov 28 16:20 .

IdM Dogtag CA issues both certificates with the corresponding CN:

To manage renewal of both certificates automatically, modify the PathModifiedby PathExistsGlob:

[client]# cat /etc/systemd/system/acme.path
[Unit]
Description= Triggers the reload of httpd
[Path]
PathExistsGlob=/etc/httpd/state/md/staging/*.lab.example.com/pubcert.pem
[Install]
WantedBy=multi-user.target

Use daemon-reload and enable the .path file again:

[client]# systemctl daemon-reload
[client]# systemctl enable --now acme.path

All the certificates in your web server are automatically rotated. The complete lifecycle of multiple domains is covered, and you have an automated and scalable solution.

Conclusion

This article has demonstrated how to set up an IdM server and mod_md client that can issue and renew certificates through the ACME protocol. Because IdM is included in your RHEL subscription, you can try to replicate this content in your lab environment without any additional subscriptions to set up your own ACME environment and mod_md client. If you are not already a RHEL subscriber, get a no-cost trial from Red Hat.