During the recent Unconference, an interesting point was made at the plenary session by Bob Spence – e-learning quality does not mean e-learning effectiveness. Unfortunately, I didn’t hear Bob’s full explanation about what he meant by this, and so I thought I’d take up Bob’s point as I see it.

When an organisation is relatively new to e-learning, there is a lot to do:

• Governance in place to manage implementation
• Skill development in instructional and visual design
• Learning to maximise the authoring tool
• Managing stakeholder expectations
• Communicating implementation
• Monitoring and measuring learning effectiveness

In the past I have blogged about e-learning quality. It is an obvious requirement. Despite the ‘consumers as producers’ trend that social media avails, as end-users of corporate e-learning, it is distracting to the learning process if the e-learning course is not a quality product. At this time, we cannot expect corporate learners to look beyond poorly designed and produced e-learning to get the learning message – I think it simply sends the wrong message – an “I didn’t care enough” message. Not to mention the threat to learning outcomes…

Quality is probably a subset of elements that make e-learning effective.

What makes e-learning effective?

We first need to start with the end in mind. Bob Spence suggested we need to ask the subject matter expert who want the e-learning course to be built “When will you know the learning was successful?” In other words, ensure to include success criteria when you are scoping an e-learning course.

Ensure the success criteria is measureable. Measurement will help you determine whether the e-learning course was effective. Some measurements will only be determined by uncovering what has changed behaviourly in the workplace.

Bob and another Unconference speaker talked about the dangers of assuming mastery at the end of an e-learning course. Until the learning outcomes have been applied and assessed in the workplace, we cannot confidently deem learners have mastered the e-learning content. Passing an assessment at the end of an e-learning course does not generally equate to mastery. To this end, Bob makes the point that staff need multiple content exposure back at the workplace, and an environment that supports learning and the development of mastery. This is where managers play an important (but not exclusive) role.

So, the bottom line, when we are developing an e-learning course:

1. At the start of an e-learning project, ensure to articulate measureable success criteria
2. Partner with corporate comms, management and leadership, and interweave the content into related policies, courseware and workplace information to help embed the learning outcomes into the workplace
3. Put mechanisms in place to measure the effectiveness of the e-learning project
4. Learn from the outcomes and use this to improve the success of the next e-learning project

I’m not a fan of a lot of the compliance e-learning I see. They are often boring and the relevance to the learner is poorly articulated. The topics are usually important; policy and safety-related. Unfortunalty, in a lot of cases, the instructional approach and media used does not reflect the significance of the topics.

Nevertheless, organisations make a decision to implement these courses (usually to reduce corporate risk), and a lot of effort is put into implementation. Staff are instructed to complete these mandatory courses. Maybe the reason for completing the e-learning courses has been well articulated, or maybe not. Nevertheless, the e-learning courses have been deemed mandatory. So, what does this mean?

Mandatory assumes someone will look up the LMS SCORM report and see that all staff members have passed the course and have completed it in the anticipated timeframe. That’s what SCORM standards are for – to provide these reports. So, how come the reporting and follow-up is often so poorly actioned?

If you are communicating “this course is mandatory” then, ensure there are processes in place to reinforce these instructions. If you don’t, you risk the following:

1. De-value the e-learning strategy
2. Put the organisation at industrial relations risk
3. Put the learner at risk
4. Put at risk any future “mandatory” courses

What is a satisfactory e-learning completion rate?

Really, mandatory e-learning cannot have any less than a 100% completion rate. Anything less than 100% has failed the meaning of ‘mandatory’ (**see below). A hundred percent of your cohort must complete the mandatory e-learning course and passed the assessment (if assessment was included).

Additionally, individual’s completion should be within about 30% of your expected completion time. i.e. use your SCORM reports to identify people who completed a 30 minute course in less than 20 minutes.

Key Messages for mandatory e-learning

1. Have a policy in place so that everyone is clear of the consequences if a mandatory e-learning course is not completed in the allocated time
2. Have a clear allocated time period in which staff must complete mandatory e-learning courses e.g. within 30 days of joining the organisation
3. Make sure managers know their role in ensuring staff have opportunity to complete mandatory e-learning courses
4. Ensure someone is tasked to run reports
5. Make sure the person who runs the reports has a mandate to a) contact those who have not completed the course and find out why (look for genuine issues)  b) action consequences for non-completions (or too quickly completed, in the case of non-assessed e-learning courses)
6. Evaluate your mandatory e-learning courses to a) ensure people are able to correctly apply their learning into the workplace after completion b) get user feedback to help inform future instructional approaches

** To achieve 100% completion rate you need to be confident that your course is tracking correctly on all operating environments, or you will have lots of staff who will say “I did complete the course, but the system isn’t registering it – it’s not my fault“. Be sure to do lot’s of testing before releasing a mandatory course.

One of the challenges of having e-learning live on the LMS is course review and maintenance. I call this the “Sustain” phase of the ADDIES model.

Often compliance-related courses and technical courses are originally sponsored by subject matter experts (SMEs). The e-learning development is commonly centralised through the Learning and Development (L&D) e-learning team who may or may not work with an external e-learning provider to have the course built.

The input of the SME is critical to the accuracy of the content, as well as the holistic ‘grass-roots’ strategy of ensuring staff are across the content (i.e. the SME is typically responsible for e-learning content, intranet content, policy, brochures and posters, workplace assessment and face to face training).

The input of the e-learning team is critical to ensure instructional approaches, learning objectives and assessment, testing, evaluation and overall project management of the e-learning course is sound.

The e-learning team will usually monitor course usage and user feedback and report the status to the SME. But the e-learning team are not subject experts, and so sustaining the course is typically the responsibility of the SME.

Amongst other things, “Sustain” is about ensuring the course content is up-to-date and accurate. It’s about ensuring the URLs are live. The SME must take responsibility to communicate timeframes to the e-learning team if a change to policy or legislation is expected. From here, a plan can be made between the e-learning team and the SME to update the course. This is a good time to include the feedback of the current course, so that the update can address both content and usability/user acceptance issues.

Below I have created a general list of tasks of a typical ADDIES lifecycle of a policy or technical-related e-learning course (click to enlarge the image). Allocate responsibilities to match your organisation’s preference for doing things. Of course, shared responsibility and partnership is ideal. However, principal responsibility for each task should be agreed at the start of an e-learning project.

If you are a member of the Connect Thinking E-Learning Academy, this table is available to you under the Resources tab, in an editable Word format, along with a host of other e-learning project tools and process documents.