For a while Garmin have been adding “wearable technology” functionality to their watches - notifications from the phone etc. I got a Garmin Fenix 3 for Christmas and had a play with the vast number...

Today I’m at my first Scala eXchange conference - the 5th annual one to be precise. The day started with the keynote session by @jessitron which was both inspiring and enlightening. The general to...

I need to write a couple of unit tests for some code to add a log entry into HDFS but I don’t want to have to rely on having access to full blown HDFS cluster or a local install to achieve this. T...

Here we are in June, some five months since the last post and I finally have some time and content to sit and write a post. In April 2013 I started working with Hadoop, the plan was to suck in ser...

All the work I have been doing with AWS has been using Python, specifically boto3 the rework of boto. One of the intentions is to limit bandwidth when transferring data to S3 the idea is to send p...

Towards the end of 2014 Amazon released the KMS service to provide a cheaper cut down offering for Key Management Services than those provided with the CloudHSM solutions (although it still uses ha...

Some time ago, back in September, I wrote a post on starting my adventures with Spark but didn’t progress things very far. On thing that was holding me back was a reasonably real world problem to ...

I’m working with a new AWS account and I am moving to testing Boto3 to use the KMS service. I needed to make sure that the AWS account and secret keys were updated so ran aws configure to quickly u...

I’m not sure if this is because we’re behind a proxy, the network has issues or my work laptop isn’t great, but for some reason the git clones very often hang during the unpacking of objects. remo...

For 18 months I’ve been working with Hadoop. Initially it was Hortonworks HDP on Windows then Hortonworks HDP on CentOS and for production we settled on Cloudera CDH5 on Red Hat. Recently we’ve bee...