Regularly when writing a shell script I find that I want to be able to pass an argument into the script but only sometimes. For example if I want the script to output to /tmp folder for the most pa...

Its reasonably easy to run a Spark job against HBase using the newAPIHadoopRDD available on the SparkContext. The general steps are, create an HBaseConfiguration create a SparkContext crea...

If you have committed some code to git (or in the current case, BitBucket) and you have made an error in the commit message (in the current case, referenced the wrong Jira ticket), all is not lost....

Following on from part 1 of this mini series. I’ve got my local environment ready to go and I have pulled down some test data to work with. The next step is to start having a look at some of the d...

I’m hoping this will be a reasonably accurate account of my play with the TfL Cycling DataSets. I’m still forming my plan, however loosely I think I want to end up with a visualisation where the b...

Now I’m writing almost all documentation in markdown then using Pandoc to convert it to Mediawiki or docx as required, I needed to finds an easier way to quickly create my tables. It doesn’t do an...

I’m having a bit of a dabble with Go, as a by product of working with Elastic Search beats. One thing I’ve been looking at today is the channels to allow two go routines to communicate with each o...

There are lots of monitoring and alerting tools out there and I’m sure everyone has there own preference on which they’re going to use. We have selected monit for simple monitoring of disk space, ...

I’ve moved to a new role where I will be doing a lot more “devops” type work which hopefully comes with more interesting subjects to start writing about. Oozie and AWS Simple Email Services The C...

Today someone showed me a handle word cloud tool called wordle. I put the skills from my CV into it and had a play with the output… I think it looks quite cool.