Cool! It's so funny my internet has been bad just recently so I took an old bash script of mine which did something similar and have been upgrading it into a proper application which exports metrics to Prometheus and Grafana: https://github.com/Noah-Huppert/net-test
I saw a comment below where some was rolling their eyes that you "complicated" stuff with Prometheus, Grafana, and Docker and how you could just use Bash scripts and crons. As I just upgraded my codebase from this more bare metal approach to this "more complex setup" I'd like to mention: there's no way you could do time series statistical analysis easily with "just a cron job and a bash script". Prometheus and Grafana are for more than just buzz words. Prometheus offers an advanced time series database which allows you to, at minimum, do more robust analysis using data techniques like Histograms. As for Grafana, it makes exploring data dead easy.
Providing users with a Docker Compose setup is also something I did with my tool and the benefits are huge. It lets me distribute a setup which relies on multiple moving parts working smoothly together. Sure I could write a whole wiki on how you should setup Prometheus Grafana and my tool, or I could distribute the setup with a configuration as code tool. Ensuring that even if someone doesn't want to use Docker Compose they can at least read my configuration as code and see exactly what I did to setup my tool.
Recently I was looking to add some metrics to a small script and I didn't want to deal with all the complexity of setting up grafana, prometheus, the push-proxy-thing (sorry, don't remember the name).
Looking for alternatives, I found InfluxDB, which was just one apt-get install away.
It comes with a web interface to create dashboards, supports push based metrics, and overall I'm loving it.
The web interface doesn't seem as powerful as grafana, but it covers most basic needs. And I think you can use it with grafana too.
>And I think you can use it with grafana too.
You can indeed; I use InfluxDB and Grafana to provide some stats on my home server. Pretty easy to set up, and Grafana just uses the Flux query language to pull the data from Influx.
The closest analogous tool people would use back in the classic cron and bash days was MRTG.
I've been tangentially looking at this as a 'beta' customer of Starlink, as the service is presently extremely variable with many small outages throughout the day. This kind of a collection of internet monitoring is super useful for the individual, but what I would really love to see is a way to federate the data in a meaningful way so that you can start to see operating patterns in the provider(s) being used.
RIPE Atlas does exactly this:
If you are on a ASN that they don't have many existing probes for, they'll send you one for free.
We do something similar at https://imup.io but we run ping & speed tests from your home PC rather than shipping you a device. We approach the situation wanting to save folks money on their internet services for degradation in addition to providing analytics.
Thanks for sharing that link for Ripe though, super cool project.
This is awesome thank you!
Use node exporter on your machines and Prometheus to scrape the metrics and you will be drowning in pretty graphs. Super easy to set up and gives you system details across all your machines, all in one place.
There are also script exporters (a few implementations) for simpler Nagios style checks and more specialised ones for getting things like Cloudwatch data (shout out to YACE[1] - with dashboards from promcat). It's really quite vibrant the ecosystem.
Does anyone still use SNMP to announce metrics/monitors?
Of course but it's still in the traditional domain of networking kit (generally). As there are many components virtualised or software defined then I think in those areas, not so much as there are alternative ways to get the metrics. SNMP is a bit long in the tooth and can be a bit unweildy sometimes, ymmv etc.
In addition to monitoring your own network, you might want to monitor your ISP's and share the results for research - a software RIPE Atlas probe is a great method for that: https://labs.ripe.net/Members/alun_davies/ripe-atlas-softwar...
I recently set up smokeping to gather some data, I can't say it was a breeze - documentation appears complete but misses some key pieces of information you have to dig for or trial/error it out.
E.g: i struggled figuring out where to put a non-standard probe
Failure modes were also quite weird... adding a slave to the mix broke my mind a little. Was never sure it worked but then didn't need it so retired it before i figured it out
Changing what you think is something trivial (e.g: frequency of data collection) invalidates your previously collected data files.
Other than that - fairly solid, did what i needed, runs without a fuss.
I've set up smokeping a number of times over decades. Always a minor pain until the most recent one...
I used a docker container, and it was a breeze! `sudo docker run -it -p 8000:80 -e WIPE=y -e TARGET="ISP;NextHop;$IP" -d dperson/smokeping`
https://github.com/SuperQ/smokeping_prober
This project is quite good for a smokeping-a-like for prometheus.
I did something similar for continuous speedtest and tracking of home Internet [0].
For simpler needs, like being able to see when my network dropped out, without needing another device... On a OpenWrt home router, I usually install `collectd-mod-ping` and `luci-app-statistics`, and have it periodically ping a couple of different hosts that will answer pings reliably. (This can double for monitoring your third-party hosting.)
Caveat: since you're probably running OpenWrt on a consumer SoHo router, I recommend sending this logging data to a cheap USB flash drive, not wearing out the router's onboard flash.
An even simpler alternative would be to keep a browser window open with the OpenWrt admin UI's "Status -> Realtime Graphs" displaying.
(If you're new to OpenWrt, a fairly recent router hardware model that's easy to re-flash with solid OpenWrt, and fairly inexpensive: https://openwrt.org/toh/netgear/r7800 )
Interesting lots of people had the same idea. I have done something similar for my home network using an Ethernet connected RaspberryPI dedicated to this task using a cronjob, the speedtest-cli JSON format, MongoDB and just some JS graphs. - very rudimentary.
One thing I found very important was to plot the scatter of PING vs. Download speed (my Upload is super stable).
Then I created the probability density function of that (using KDE) and used it to get a better deal from my network provider - I could prove they were selling me a slower network most of the time - they now allocate 70Mb/s on a 50Mb/s deal so just that they hit the 50Mb/s most of the time.
It also allowed me to clearly identify some outlier situations where my network suddenly became 'bad' and I could show evidence of it being bad over time.
Certainly a project that should see more traction!
Get a daily email with the the top stories from Hacker News. No spam, unsubscribe at any time.