Tuesday, January 15, 2013

Simple python script to access jmx via jolokia

I've never been much of a python guy. Now there's this production environment where we've got a few basic things like bash, ruby (since we're using puppet) and python. Our Java based software has a nice JMX interface with a bunch of lifesavers in it and each JVM has the jolokia agent running (http/json access to jmx) so we can access this goodness fairly easily from scripts and/or the shell. So far what we'd be doing would be something like this:

$ curl http://server:port/jolokia/read/java.lang:type=OperatingSystem

This would return a ton of json and is very hard to read. Later we found out there's this niftly little python module and so we get nicely (and readable) json output by just piping it trough python:

$ curl http://server:port/jolokia/read/java.lang:type=OperatingSystem | python -m json.tool

So far so good. Now because I'm lazy and don't like typing or even copying these urls into a shell, a bash script was born, which was a collection of calls like you see above. You'd source the script and use the functions contained in it like they were system commands. Pretty nice, I thought. Until we ran into a problem that needed checking if something had a certain state and then changing that state, thus actually working with json in a bash -- good luck with that!

Since we've already used that nifty json.tool I ventured into using python for this and it turned out it's pretty damn awesome. We can now call a jolokia url, work with the return value and do some more calls based on that. I did that using the subprocess module and calling the shell functions but that's ugly. So I went ahead and ported the shell script to python.

What you see below is a python script that can be used either as a module or directly from the commandline. Thanks to python's cool inspection/reflection capabilities (As a java guy I'm used to that anyway), you can extend this script by simply adding another url and correspondingly named function. If something's off it tries to print an error as good as possible (includes argument names, how cool is that!) and if everything's right it just works as you'd expect.

I hope there are other jolokia users out there who will find this useful. And to the python wizards: Let me know if and how I can improve on this script (especially the main function).

So yeah, I've never been much of a python guy -- until now. Enjoy my first python script!

Loading ....

Tuesday, January 8, 2013

Pretty Capistrano output

I recently started tinkering around with capistrano to automate a few things like doing checks and generally make my life easier in terms of managing a bunch of linux machines. While capistrano is really cool, it's output is all over the place and I find that very irritating. Maybe it's just me but I don't like non-aligned repeating Strings. Take this output for example:

 ** [out :: borked.swisstech.net] puppetd (pid  14395) is running...
 ** [out :: carmine.swisstech.net] puppetd (pid  14433) is running...
 ** [out :: mandelbrot.swisstech.net] puppetd (pid  2835) is running...
 ** [out :: magi.swisstech.net] puppetd (pid  28830) is running...
 ** [out :: enchilada.swisstech.net] puppetd (pid  4455) is running...
 ** [out :: titan.swisstech.net] puppetd (pid  30098) is running...
 ** [out :: kronos.swisstech.net] puppetd (pid  27332) is running...
 ** [out :: serenity.swisstech.net] puppetd (pid  11072) is running...
 ** [out :: ackbar.swisstech.net] puppetd (pid  17522) is running...
 ** [out :: r2d2.swisstech.net] puppetd (pid  15535) is running...
 ** [out :: seraph.swisstech.net] puppetd (pid  24193) is running...
 ** [out :: box.swisstech.net] puppetd (pid  23061) is running...
 ** [out :: fermi.swisstech.net] puppetd (pid  8380) is running...
 ** [out :: gossamer.swisstech.net] puppetd (pid  28875) is running...
 ** [out :: marvin.swisstech.net] puppetd (pid  17977) is running...

The above is one of the easier to read examples. There are many more that are much worse to read than that (but too big to paste into the blog here).

So when looking for a solution I came across a blog entry called Streaming Capistrano that described how to tail from logfiles. Of course I had to try this and the completely unformatted output became apparent again.

So what to do? I already had half of the solution in front of me thanks to the post linked above. It didn't put the hostname in front of each line, only the first of a bunch so we need to iterate over the lines in the data variable. Plus to align everything we need to calculate the longest hostname in use and fill shorter hostnames with spaces before we print the data itself. Luckily, Ruby Strings have some pretty cool helper methods like ljust, rjust, center and many more so this couldn't be easier to solve.

A little bit of tinkering later I made that thing into a function and thus reusable in my entire capfile. So instead of 'run' I now call 'arun' (aligned-run) and that's it!

So here's the same output but neatly aligned. Much easier to read, isn't it?

 ** [out :: borked.swisstech.net    ] puppetd (pid  14395) is running...
 ** [out :: carmine.swisstech.net   ] puppetd (pid  14433) is running...
 ** [out :: mandelbrot.swisstech.net] puppetd (pid  2835) is running...
 ** [out :: magi.swisstech.net      ] puppetd (pid  28830) is running...
 ** [out :: titan.swisstech.net     ] puppetd (pid  30098) is running...
 ** [out :: enchilada.swisstech.net ] puppetd (pid  4455) is running...
 ** [out :: kronos.swisstech.net    ] puppetd (pid  27332) is running...
 ** [out :: serenity.swisstech.net  ] puppetd (pid  11072) is running...
 ** [out :: ackbar.swisstech.net    ] puppetd (pid  17522) is running...
 ** [out :: r2d2.swisstech.net      ] puppetd (pid  15535) is running...
 ** [out :: seraph.swisstech.net    ] puppetd (pid  24193) is running...
 ** [out :: box.swisstech.net       ] puppetd (pid  23061) is running...
 ** [out :: fermi.swisstech.net     ] puppetd (pid  8380) is running...
 ** [out :: gossamer.swisstech.net  ] puppetd (pid  28875) is running...
 ** [out :: marvin.swisstech.net    ] puppetd (pid  17977) is running...

Now I'm really not a ruby hacker. This works for me but if the more experienced rubyist sees anything wrong with the above code please let me know.

I think there are two potential problems with this code:

  • As far as I can tell, it may happen that the first few lines aren't aligned because I don't analyse the length of the hostnames upfront but do this on the go. But I think I can live with that.
  • The statement length = [length, host.length].max creates a list in the background which may be a bit too much overhead for many hosts and lots of data going trough this function but so far no problems.

Why isn't that the format of capistrano anyways? The output is so much easier to read!

Hostnames for the anonymized examples above were generated using computernamer.com

Wednesday, January 2, 2013

Quality loss when saving as JPEG

So recently an article was linked on reddit about a bunch of photography myths. Number 7, called "Saving a JPEG multiple times degrades quality" which was apparently proven to be wrong disturbed me most and so I did a quick test on my own.

Further below are Three images. The Original, the one saved 30 times and an XOR of the first Two showing the differences. If you toggle between the first two you can see quite a difference in the color on the car and several other details. Running any more tests at lower than 100% quality is pretty much a waste of time once you see these results.

Here's how I constructed these images. Feel free to run your own test, it's really simple. All you need is a linux machine and ImageMagick. The code below will do the following:

  • save image0 to image1 at quality 100%
  • save image1 to image2 at quality 100%
  • ...
  • save image29 to image30 at quality 100%
  • calculate the difference (XOR) between image0 and image30
Loading ....