In the past year I moved our teams entire infrastructure monitoring from Nagios/collectd to Prometheus. The amount of visibility into our infrastructure it has provided that we didn’t have before has been invaluable.
We host a bunch of small WordPress and other custom built websites (along with a couple very large sites) that we use the Blackbox exporter to monitor the response code and SSL Certificate status. I wanted to be able to keep all the sites we monitored in two files without having to modify the prometheus.yml every time I needed to add or remove a site which could require a reload of Prometheus.
I ended up using the file based service discovery for this. I had to dig quite a bit to find an example (which I think I found in a Github issue). Someday I want to expand this to use proper service discovery (like we do for all the other exporters) but I wanted something simple to start.
Below is examples how I have this setup in the Prometheus and Blackbox exporter configs.
1) Add the blackbox exporter job to the prometheus.yml file:
2) Then we need to configure the Blackbox Exporter on how to handle our two config files (one for http one for https):
3) Finally our last two configs. These are the actually lists of sites that the blackbox exporter will be monitoring.
That should be it. Any changes you make to either of the http_2xx.yml or https_2xx.yml config will automatically be picked up by Prometheus based on your scrape_interval setting.
Feel free to give me a shout if you have any questions or issues.
Hi Seth,
First, A big applause to your amazing work.
I Just struggling to monitoring the https endpoint through black box exporter. i can able to monitor the http endpoint without any issues. The ssl configured websites are running UP but prometheus still firing in sites are down.
I followed your instructions but it does not help me. I do not know where and what is missed.
Thanks in advance
Karan
Hi Karan,
You could try setting this in your config: preferred_ip_protocol: “ipv4”
Like so:
https_2xx:
prober: http
timeout: 5s
http:
method: GET
no_follow_redirects: false
fail_if_ssl: false
fail_if_not_ssl: true
preferred_ip_protocol: “ipv4”
I think I had to add that at some point because of some weirdness with IPV6.
Seems the comments killed the formatting. But I also updated the post with that additional option.
I was trying to simplify job configuration by abstracting the module to be used by blackbox exporter and your post came really handy! Thank you!
Glad to hear it!