E-learning surveys - Lies, damn lies and statistics.

Last month we reviewed Survey Money and as part of this review we set up a survey on future e-learning trends for our community to take part in. Whilst our ours was just a small sample of a few hundred people from our community of readers we thought we would compare it to the original E-learning Guild survey from which we took our questions to see how things compared. We then also compared the findings to the latest CIPD survey on learning developments which highlighted the difficulty of analysing and interpreting surveys.

The questions we asked and the results were as follows:

How much will your organisation increase or decrease the following activities over the next 12 months. Will you increase significantly, increase, neither, decrease, decrease significantly (answers in order from left to right).

(Answers rounded to nearest percentage.)

We took these questions from the E-Learning Guild survey so we could compare our results. We would emphasise though that the E-Learning Guild survey is in our view the best and most thorough e-learning survey. The Guild surveys over 20,000 of its members and provides very structured feedback by size, industry, etc. Ours was a smaler snapshot, mainly used to demonstrate Survey Monkey, but we thought it would be fun to compare.

Overall the findings were very similar, for example:

In the 2006 E-Learning Guild survey 77% said they would increase or significantly increease the use of e-learning to train customers and partners.  In our survey it was 79%.

In the 2006 E-Learning Guild survey 79% said they would increase or significantly increase the use rapid e-learning. In our survey it was a little lower at 73.1%.

There were two questions where our results differed significantly, these were:

Use of authoring tools - In our survey 73% said they would increase or significantly increase the use of authoring tools compared to 64% in the Guild survey. This might be due in part to the improvements that have been made in authoring tools, making it ever more possible to do great work in-house.

Use of e-learning to support informal learning - In our survey 80.2% said they would increase or significantly increase the use of e-learnign to support informal learning compared to 69% in the Guild survey. This might be due in part to developments in informal learning and the increasing use of web 2.0 tools and technologies. In our follow up questions, serveral responses focused on how to bring these tools into the organisation and engage learners, so we can expect to see more in this area.

One to watch: Use of e-learning to train customers and partners: 53% expect moderate increase, 23% a significant increase

Could this be the sleeper trend of 2010? E-learning for channel partners is well established in the US, with many organisations focused on platforms for delivering and communicating with channel partners – LogicBay is a great example. In the UK it’s less well established, but has great potential to grow. One to certainly watch, especially if combined with some smart use of mobile learning / web 2.0 tools to keep sales and channel partners up to date. The growth of e-learning for customers is going to be interesting and another one to watch closely in the UK.

CIPD Survey

The latest CIPD Learning survey had us worried on first glance.  It concluded that 'Few feel [e-learning] is the most effective learning and development practice (7%)' and less than half are using more e-learning. This would seem to contradict the findings of other surveys. However, when we looked at the sample we found 55% of the survey responses were from organisations with less than 1,000 staff. 77% were from organisations with less than 5,000 staff. It is not surprising that smaller organisations find e-learning less effective. At Kineo we work with a lot of global corporates and e-learning is very effective at delivering learning quickly and cost-effectively to a global audience. The most recent e-learning programme we developed was in multiple languages for delivery to 150 different countries. It's an old truth but it's still accurate: e-learning works on scale.

It would have been far more helpful if the CIPD had looked at the needs and responses of different organisations. To generalise the findings on e-learning effectiveness seems a nonsense when over half of the respondents have less than 1,000 staff. Hopefully future reports will give us a better breakdown.

Also as Clive Shepherd recently commented "what sort of e-learning are they talking about - self-paced, self-study lessons? live online events using virtual classrooms? collaborative online learning? It is impossible to have one measure for the effectiveness of all three, because they have almost nothing in common except for the fact that they are delivered using technology."

We all need to be careful of surveys or "Lies, damn lies and statistics" as Mark Twain once said. Almost 62% of people agreed with him at the time.