Over the last couple of weeks I’ve written about processes around engaging and working with e-learning providers. Refer to:

What to include in an e-learning Request for Quotation

Milestones of a typical e-learning project

Today I thought I’d share my observations about why some e-learning projects fail. I’ll use the themes of cost, quality and time.

E-Learning cost ‘blows out’ because…

• Requirements have been poorly scoped in the first place
• Stakeholders have been consulted late in the project, resulting in scope creep

Solution: Develop a scoping document for different kinds of e-learning that will guide requirements gathering before engaging an e-learning provider. Make sure this document is signed off by all project stakeholders.

E-Learning quality disappoints because…

• The wrong e-learning provider was chosen in the first place
• Stakeholder expectations are unrealistic for the budget allocated
• End user requirements and expectations were not properly investigated
• User acceptance testing was poorly conducted prior to ‘go-live’

Solution: Ensure you evaluate examples of comparable e-learning topics from potential e-learning providers, and contact their references. Examine the learning and accessibility needs of your end users carefully so as to not disappoint and frustrate. Formalise your UAT process to ensure its rigour.

E-Learning project timeline ‘blows out’ because…

• The internal project team (including SMEs) underestimated the time required of them to review each stage of the developing project (see Milestones of a typical e-learning project)
• Unforeseen IT internal operating environment issues (e.g. issues with SCORM standards between the developed course and LMS)
• Internal project team and/or e-learning provider project team flutuations

Solution: Ensure the project timeline is realistic in the first place. Raise the issue of timeline slippage early and rectify quickly to avoid a domino effect. Make sure e-learning providers outline how they will manage staff changes.

Of course there is a lot that happens within each of these listed causes. The more experienced we become in managing e-learning projects, the better we become at implementing strategies to mitigate project risk. Be sure to conclude each project with a closure report to record lessons learned for those who will follow your footsteps.

For information and tools, contact us to learn how the E-Learning Academy can support you during your e-learning project.

Last week I blogged about what to include in an e-learning request for quotation (RFQ). A RFQ helps make clear the requirements of an e-learning course for development.

Meeting requirements and expectations during an e-learning courseware development project is a 2-way street. The role of the client is to keep objectives clear, manage the internal stakeholders and project processes. The role of the e-learning provider is to listen to the client, meet their needs, and add value to the project experience and outcomes along the way.

Creating an e-learning course is a multidisciplinary, multimedia production. Like any production, it is dependent upon the quality of the outcome that came before it. From start to finish, there are clear milestones where the e-learning provider must seek sign-off from the client to help ensure all is correct before the next part of the process proceeds. The client has responsibility to complete a through check before signing-off the milestone.

E-Learning courseware milestone sign-offs

Let’s take up the project assuming a RFQ proponent has been chosen. There are typically six milestones that require the client to review carefully and sign off.

A. Design brief: This high level brief should reflect both the RFQ inclusions and details discussed at the kick-off meeting. It should outline the design approach for what s being taught, the assessment approach, graphic style, interactivity and animation to be used, video and/or narration, navigation inclusions, authoring tool and understanding of the client’s technology environment (standard operating environment – SOE). It should also include a few template mock-ups called GUI (graphic user interface). These are visual treatments that should reflect a little of your corporate look/feel, but should also be mindful of the learner experience. Get GUI sign-off from Corporate Comms before signing off the Design brief.

B. Test course: From the design brief the e-learning provider should create a small test course reflecting all of the agreed navigation, interactivity and assessment type. Test the course in your LMS and check all the functionality works. Test the assessment tracks correctly and the course completes. Then sign-off.

C: Storyboard: This is perhaps the most complex milestones, as it requires the client and e-learning provider to come to consensus about how the course content will be conveyed to the learner. Clients new to developing e-learning will often concentrate on the source content, whereas a good e-learning provider will be designing for learning experience and engagement. It requires give and take. Where possible, go through the first draft in person so that each party can better convey their perspective. Several iterations of the storyboard may be required before the document is ready for sign-off. Remember, this storyboard is in document form, the next step is production using multimedia. so, any alteration to the storyboard once production is underway is usually considered a project variation. It may cost you.

D. Narration script: If the narration is not verbatim to the text on screen, or it may be a video script, then these also need to be sgned off prior to studio or video production. You are looking for a script that is concise, compelling and reflects your culture.

E. Alpha course: This is the first draft of the produced e-learn. You will need to run a user acceptance test (UAT) internally to check both the technical aspects of the course, and the course content. Expect to find some minor issues, which will need to be documented as a change request. if you receive an alpha course with lots of issues, send it back to the e-learning provider for another quality assurance review – it will take too much of our UAT team time to notate the issues. Where possible, run your alpha test in your LMS so that tracking is also being tested. Sign off the change request before returning it to the e-learning provider.

F. Beta course: This course should be almost perfect. Use your change request data to check that all items have been actioned. Check your LMS tracking once more. You may find one or 2 issues. If they are minor (e.g. punctuation issue), you may decide it is not worth asking the provider to publish yet again (which means more testing for you). It is very difficult o get everything 100%. perfect. If it does need updates, document them in a change request and repeat the beta testing process.

Your course should now be live on the system…don’t forget your comms plan…

These milestone sign-offs are just one component of successful e-learning project management. For information and support, including a course and tools, why not join the E-Learning Academy. I also run E-Learning Project Management workshops.

During the recent Unconference, an interesting point was made at the plenary session by Bob Spence – e-learning quality does not mean e-learning effectiveness. Unfortunately, I didn’t hear Bob’s full explanation about what he meant by this, and so I thought I’d take up Bob’s point as I see it.

When an organisation is relatively new to e-learning, there is a lot to do:

• Governance in place to manage implementation
• Skill development in instructional and visual design
• Learning to maximise the authoring tool
• Managing stakeholder expectations
• Communicating implementation
• Monitoring and measuring learning effectiveness

In the past I have blogged about e-learning quality. It is an obvious requirement. Despite the ‘consumers as producers’ trend that social media avails, as end-users of corporate e-learning, it is distracting to the learning process if the e-learning course is not a quality product. At this time, we cannot expect corporate learners to look beyond poorly designed and produced e-learning to get the learning message – I think it simply sends the wrong message – an “I didn’t care enough” message. Not to mention the threat to learning outcomes…

Quality is probably a subset of elements that make e-learning effective.

What makes e-learning effective?

We first need to start with the end in mind. Bob Spence suggested we need to ask the subject matter expert who want the e-learning course to be built “When will you know the learning was successful?” In other words, ensure to include success criteria when you are scoping an e-learning course.

Ensure the success criteria is measureable. Measurement will help you determine whether the e-learning course was effective. Some measurements will only be determined by uncovering what has changed behaviourly in the workplace.

Bob and another Unconference speaker talked about the dangers of assuming mastery at the end of an e-learning course. Until the learning outcomes have been applied and assessed in the workplace, we cannot confidently deem learners have mastered the e-learning content. Passing an assessment at the end of an e-learning course does not generally equate to mastery. To this end, Bob makes the point that staff need multiple content exposure back at the workplace, and an environment that supports learning and the development of mastery. This is where managers play an important (but not exclusive) role.

So, the bottom line, when we are developing an e-learning course:

1. At the start of an e-learning project, ensure to articulate measureable success criteria
2. Partner with corporate comms, management and leadership, and interweave the content into related policies, courseware and workplace information to help embed the learning outcomes into the workplace
3. Put mechanisms in place to measure the effectiveness of the e-learning project
4. Learn from the outcomes and use this to improve the success of the next e-learning project

Here is a list of questions to ask an e-learning provider before you engage them. Thank you to Zoe Karathomas for allowing me to share our combined thoughts:

1. Have you worked with clients of a similar size/industry to ours? (and can I have a list of recent referees to contact?)
2. Have you developed similar materials and content to ours? (and can we see recent demos?)
3. Can you explain to us your instructional design methodology?
4. Will all development be done locally? (off-shoring may be cost-effective - but you need to ensure the communication channel is sound)
5. How do you assure our IP and source files are safe?
6. What strategies will you put into place to future-proof the finished e-learning course? (e.g. do you have the ability to export the course to multiple file formats for use on mobile devices?)
7. Is the e-learning authoring tool you use bespoke, or is it a proprietary tool? (if it is bespoke, this will mean you will need to return to the provider each time you need an update, or you will need to purchase their tool)
8. What guarantee can you provide to ensure the project will be delivered on time and to budget?
9. What is your project management methodology? (in particular, check the frequency of Quality Assurance checks)
10. How frequently will you hold project briefings with us? (and how will you raise and escalate issues?)
11. Do we incur a penalty if we don’t deliver our reviews by the documented due dates?
12. Who in your team will be on the project? What are their qualifications? (and how quickly can their role be filled, should they leave?)

The best way to manage e-learning projects is to have a methodology in place yourself that you can dictate to the e-learning provider. This should be discussed and agreed upon prior to formal vendor engagement.

If you would like assistance in developing an internal e-learning project management methodology and governance, stay tuned. I will be launching a new workshop on this topic in March.