Monday, 2 February 2015

Playing with elephants - Hadoop?

People who know me know that I'm interested in databases and SQL. I continue to describe myself as an Analyst rather than a Developer or DBA, but I also think it is useful to have a basic understanding of the characteristics of the tools one might be using. I'm busy at the moment but I thought that it was high time I "nailed my colours to the mast".
I intend to start looking at Hadoop before the end of the year.
I know that may seem like a long way off, but it is getting closer all the time.

Here is one of the articles which grabbed my attention: http://www.sqlservercentral.com/articles/Hadoop/99135/

There is nothing like committing that you are going to do something as a bit of encouragement.

Why do I want to look at Hadoop?

  • First of all, I want to do it as an excuse to "brush up my Unix" (sorry Linux)
  • That makes it an excuse to buy and use some new (to me) hardware.
  • It will also be a reason to collaborate with an old acquaintance of mine.
  • And I find the idea of "map reduce" intriguing and trying it out is the best way to learn.


As I said, I'm busy, so that's it for this week!

Monday, 26 January 2015

Day to day plans - Having a dynamic To-Do list

I've come to regard myself as being like Winnie-the-Pooh "a bear of little brain". I like to give myself a task, settle down to it and get on with it without distractions. Sometimes the task itself can be quite complicated though.

In recent posts I've shared a view of "Strategic" and "Tactical" plans with you. I know you haven't seen what is inside them but I find having them enormously useful. That brings me to the next level down: the day to day, dynamic To-Do list.

Having a dynamic list of the tasks I have to do is enormously useful. If I'm managing other people doing things, then I need to know what they are doing too. This is what the classic project plan is all about. Even a small team needs a shared plan. I used to create them using spreadsheets but now I use a special purpose tool.


The picture above shows screenshots from today's dashboard. The tool I use is PBWorks Project Hub (http://www.pbworks.com/). I use the "Freemium" version, and I find it adequate for my needs. It does most of the things I would have been doing with a spreadsheet and it does some other things too. 

It is very good at doing the boring Project Office things like bugging people (including me) when they are due to be doing something today. It is also good for collecting reports of progress made or problems encountered. All that has to happen is that people have to get into the habit of making one-line notes on the task they are working on each day.

I have a geographically dispersed team. The two of us exchange notes and have the occasional phone call, but the project dashboard gives us something that we can go back to for status information without a lot of cumbersome bureaucracy (which would usually devolve to me!). The only downside of this approach is that we are using a cloud service which requires internet connectivity and could be vulnerable. That means that I take what I think are appropriate steps to have the critical information stored in some other form elsewhere.

Monday, 19 January 2015

What do you do when things go well? Make a new plan!

A little while ago I shared my “Tactical Plan” with you. Things have gone well! So well that some of the projects have been completed much sooner than I expected. Some of this is down to good luck and having estimates which were not much better than guesses. The question is: what to do now?

I expect we are all (painfully) familiar with the situation where a project is slipping behind the intended schedule. Sometimes the reasons are the same as the reasons for my success: luck, inaccurate estimates and sometimes “force majeure”. After the inevitable struggle to get back on track by cracking the whip, the response is usually to re-plan. That is exactly what I have done in response to my recent success.

Here is the redacted version of my latest tactical plan. In structure it is identical to its predecessor. The changes are in the bits you cannot see.

Let me tell you about what is in the updated plan (without giving any secrets away):
  1. The objective was not achieved last time (but that would have been a miracle). So that has remained unchanged.
  2. The “New Product” development is still (more-or-less) on track, so that remains unchanged from last time. Basically it is – “Keep working away on the new product!”
  3. The “Marketing” project from last time has completed. It’s objective was to create something new. That has been done. Now I have a new project to use what was created by that project as part of a regular activity. I still call it a “project” because it certainly hasn’t become “business-as-usual” yet. I hope it will do eventually.
  4.  The “Administration” project from last time has been completed. It was necessary, and it is making things work more smoothly, but it is complete and there is no need for follow-up.
  5.  The completion of the “Administration” project has created an opportunity to start something new in the “Marketing”. That has already thrown up some interesting ideas, but the Tactical Plan is reminding me to focus on my current objectives. The new stuff can be considered for the next plan.   


The tactical plan is part of the project wiki (I may show you a little more of that in the future). It is visible to all the contributors and I have a printed copy above my desk, a little to my left, as I type. It is great as a means of reminding us all (especially me) what we have agreed we are concentrating on.  

Monday, 12 January 2015

Do we learn from our mistakes?

This is the time of the year when a lot of us spend a little time reviewing how well we did last year and what we plan to do this year. As you may have noticed, I’m making my plans a little more obvious this year. Before we get too committed to the planning process we should ask ourselves:

Do we learn from our mistakes?

Because if we don’t, or at least if we don’t try to learn from our mistakes, then the activity is essentially futile and we would be better doing something else instead.

I think that the IT industry is particularly bad at this. We have very little sense of history. We claim to be making progress – but are we?

Here is one person’s attempt to write down an outline of that history. I can’t say I agree with it entirely. In fact I haven’t really tested it critically. But do think that it is a worthwhile effort to present one view of that history.

Why do I think the IT industry is bad at learning from it's mistakes?

Why do I think the IT industry is bad at learning from its mistakes? Basically, I think this because I frequently get a feeling that “I’ve seen this before”. Now there is no doubt that a lot of progress has been made. A lot of that progress is down to “Moore’s Law” which has enabled whole industries to sprout, bloom and flourish.

The reducing cost of hardware in general and processing power, storage and communications has enabled things to happen which while they were conceivable, were hardly practical just a few years ago. This blog and the thousands like it are an example of that.

But, if you look at various internet forums as I do, you will find recurring themes which I am going to share with you here and I may pick up as topics at some time in the future. The thing is that many of these meta-topics have been running for donkey’s years!
  • How do we write “requirements” so they are understood by the people who want the system and also by the people who are going to build the system?
  • How do we create things so that people get what they are expecting?
  • How do we estimate how long it is going to take us to do (practically anything)?
  • How do we manage a project so that it comes in “to specification, on time and on budget”?
  • How do we prevent “silly little bugs” creeping into the system?


Maybe you have some ideas for other topics in the same area, or maybe you have some solutions to some of these conundrums.

Monday, 5 January 2015

Wheels within wheels, plans within plans, but we do need action!

A week ago I shared the front page of the strategic plan for my business with you. Plans are all very well but what we really need is action, but of course we need controlled action which is moving us towards the objective.

With this in mind, I created what I've described as a "Tactical Plan" which I'm going to share with you. The tactical plan sets out an objective and some projects I am going to be working on for the first 3 months of 2015. I've included a redacted plan below. I first heard the word "redacted" in association with US Government documents. I think I would probably use the word "censored" instead. By-the way the Russian word for "editor" is "redactor" (редактор). I'm not sure if that is one of the little tricks laguages play on us, or whether it tells me anything about the Russian attitude to editing.



That really is all there is to the Tactical Plan. It is pinned to the wall in my office and it reminds me what I am achieving at the moment. It helps me to focus and keeps me from getting distracted. Of course, there are more detailed plans as well.

As you can see from the redacted plan, I have an objective, an end date and three projects. The three projects are addressing areas which I know need work: Product and Marketing are hardy perennials and in this case I felt it was time to do a bit of spring-cleaning in the thing I use for the detailed plans and tracking. I may show you that some time in the future.

Tuesday, 30 December 2014

It's always good to have a plan!

This is the time of year for reviewing what we have  done and thinking about what we are going to do. It is the time for making plans. The month of January is supposed to be named after the Roman god "Janus" (although there is some dispute about this)
Janus is depicted as having two faces and was the god of doorways and gates. He looked both inwards and outwards, forwards and back. In my opinion Janus is a good character to bear in mind when writing plans and reviews.

In the middle of 2014 I decided that it was high time that I wrote a "Business Plan" for my little business. I've done this sort of thing before for other people, but it feels a bit different when it is for yourself.

As it says at the bottom of the front page: 

"Duhallow Grey Geek started without a clear business plan. This document rectifies that. It summarizes the current situation and identifies options. It identifies how tactical plans will be created and provides an outline for the next one to two years."

Before you start writing (or even researching) any document, it is a good idea to decide who you are writing it for. In this case the answer was: for ME! That's right - for myself! Of course, I may want to present it to potential investors or business partners but I am the person making the largest investments in terms of effort, time and life. If I think I am going to be wasting my time and effort, I want to find out now, so I can do something more rewarding.


There are plenty of templates for what a business plan should contain, so I won't share the detailed table of contents. In fact I found myself adding things to the standard contents. Some of the things I included (which you may, or may not, think are "standard") are:

  • Motivation - Why was I doing this? Why was I excited about it?
  • The current position of the business - In terms of product and sales.
  • Product - What is the product? 
  • Market - Who buys the product?
  • Industry - What is happening in the industry I'm involved in? Where is the growth?
  • What resources and capabilities do I have access to?
  • Constraints - What are the restrictions that I want to apply to the business?


While I was mapping out the contents, I made a list of the questions I wanted to answer and used them as the basis for research. In the end I produced appendix material on:

  • The economics of the industry I am working in (on-line training material)
  • Sales - past performance and future projections
  • Marketing options
  • Alternative sales channels
  • Successful competitors

Predicting future sales is always difficult. In the end, I didn't try and make predictions. Instead I projected the past performance into the future and then identified the ways I could improve it. I also identified high and low levels which I could use to plan potential investments. 

The "Strategic" document I've produced, documents the facts and identifies the options. On the basis of the information available, I've picked some things I am going to do (in fact, I've started doing them already) and created what I term a "Tactical Plan" for a fixed term. I've going to "do the actions" in the Tactical Plan and monitor the results. Towards the end to the period of the plan I will review the results and decide what to do next. 

Wash, rinse, repeat....
  
Is it all going to work? I don't know. What am I going to do? That would be telling! Keep watching and you'll find out.


Thursday, 13 March 2014

Mind maps and SQL


In a recent discussion on LinkedIn, I mentioned that I use Mind-mapping. I generally prefer pen and paper or pen and white-board, because I don't like to be constrained by what the tool wants to do. I do use Freemind sometimes, and I said in the discussion that I what I sometimes do is:

  1. Create the mindmap freehand
  2. Transfer that into Freemind - which consolidates the thinking and gives me something with is tidy and easier to maintain, and then
  3. "Print" the map to pdf - which is easy to distribute and can form the basis of discussion at a distance.

I thought I would illustrate this with an example:

A recent project of mine has been creating an introductory course titled "SQL and Relational Databases for analysts".

The objective of the course was to give a basic understanding of SQL to Business and Technical Analysts.
It was intended to use MS SQL Server, but not be a course on SQL Server. The reason for this was to make the skills learned as portable as reasonably possible.

As it was intended to be an introduction, certain things I would like to have included (like UNION, HAVING and the database catalogue) didn't make the cut on grounds of keeping the size of the course down.

Anyway, the content of the course was documented in a mind-map which was then discussed with people in different places over a short period. I've attached the final version of the mind-map.
Everything in the mind-map (with the exception of the "title block" in the middle) was produced in Freemind.

The mind-map proved to be useful for agreeing what the content and structure of the course was going to be and then as a reminder of scope during the development of the course.

Here's mind-map (it was intended to be printed, if that ever happened, on A3 paper).