ELearning/Course design/Front-end analysis
Introduction
Well-defined requirements at the beginning of a design project are much more likely to result in a solution that meets its goals. Joe Harless (1973), the originator of the term, said that front-end analysis is about money, first and foremost. "It's about how to spend money in ways that will be most beneficial to the organization and the performers in that organization. It's also about avoiding spending money on silly things like instruction when there is no instructional problem; or training everybody on everything when you can get by with training fewer people on fewer things or ... sending out a checklist."
Rossett (2009) offers five assumptions behind the need for upfront analyses:
- Study prior to action improves the quality of the effort and the results.
- Incorporation of several sources yields a better program than an approach that relies only on the perspectives of one source.
- Data, broadly defined, are critical to figuring out what to do.
- A systematic approach to analysis is good for the organization and its people.
- A systematic approach to solutions is good for the organization and its people
In circumstances where instruction is called for to address a performance issue, a thorough needs analysis is called for, also referred to as performance analysis. By conducting the analysis we can be sure that it is indeed a knowledge, skills, or abilities issue and not a systems, policy, or managerial one.
Task analysis seeks to define the procedural steps, knowledge, and skills necessary to perform a repetitive task, based on observations of performance and interviews with performers.
Content analysis investigates the knowledge necessary to address the performance issue or, in the case of educational courses, the scope and depth of content to be included from within a body of knowledge.
Needs analysis
Any time there is a business need to be addressed, a needs or performance analysis is called for before any design tasks are contemplated. Here we describe the essential basics of needs analysis, and recommend further investigation into needs or performance analysis and consulting. Both address the people issues of working with internal and external clients that instructional design does not.
"Through needs analysis, we can identify deficiencies in knowledge, skills, and attitudes that can be met through education or training" (Sink, 1992). Without this analysis, we risk failing to address the true needs which may lie outside these realms, such as inappropriate policies and procedures, inadequate documentation or tools and equipment, an unproductive physical environment, motivational strategies, or management styles.
Business needs can be either problems or opportunities. A problem exists when there is actual or probable financial loss, decrease in production, or a decrease in job satisfaction. Opportunities provide the potential for financial gain, increased productivity, or increased satisfaction of employees.
Needs analysis is the systematic attempt to determine the difference between an ideal situation and the existing one, and to identify the sources of that difference. Note that we say "sources" because nearly all subpar situations have multiple causes. Any real solution will need to address, or at least consider, them all. Education and training can impact employee knowledge, skills, and attitudes but not the other problem sources.
Performance gap analysis
We begin the needs analysis process by defining the gap between "where we are" (the present level of performance) and "where we want to be" (the desired level of performance). Both the current and desired states should be stated in terms of quality, quantity, time, and cost. This means examining each category (e.g., quality) and describing it as accurately as you can. These descriptions begin simply but grow more complex as you add quantitative and qualitative measures. Categories in need of examining and different measures are addressed below.
Systems analysis
In diagnosing performance issues, it is useful to examine the systems and "forces" operating within the organization. Organizational systems constitute the context in which employees perform and lie outside the training realm. Without addressing or at least considering issues that arise during the analysis, training programs are more likely to fail. Organizations can be seen as composed of four major subsystems: human/social, technical, information and decision making, and structural (Chevalier, 2008). The subsystems and their various performance factors, which may constitute potential problem areas or strengths, are shown in Table 1 below. Additional factors may be in play depending on the organization.
| ||||||||||||
Force field analysis
Force Field Analysis is a general tool for systematically analyzing the factors found in complex problems. It frames problems in terms of factors or pressures that support the status quo (restraining forces) and those pressures that support change in the desired direction (driving forces). A factor can be people, resources, attitudes, traditions, regulations, values, needs, desires, etc. These wider issues as well as each of the subsystems need to be reviewed to see how their various factors are driving or restraining any effort to close the identified organizational performance gap. This is referred to as "force field analysis". As a tool for managing change, Force Field Analysis helps identify those factors that must be addressed and monitored if change is to be successful.
|
|---|
| 1. Force field analysis |
Driving forces. What are the factors or pressures that support change in the desired direction? What are the relative strengths of these forces? Place these driving forces on the chart on the Force Field Analysis diagram as labeled arrows with the length of the arrow reflecting the relative strength of each force. What are the inter-relationships among the driving forces?
Restraining forces. What are the factors or pressures that resist the proposed change and maintain the status quo? Represent these forces on the diagram as you did those for the driving forces. What are the inter-relationships among the restraining forces?
As we see, there are many factors beyond knowledge and skills that affect organizational performance. While it is outside the scope of instructional design to address the other problem causes, we need to be able to recognize them before instruction is selected as the intervention of choice. The other issues are generally addressed through organizational development, behavior engineering, and management consulting efforts. Chevalier (2008) provides a number of performance analysis job aids for this purpose.
Types of needs
Morrison, et al. (2011) identify five types of needs and data sources.
- Normative needs compare the organization or target audience to a peer group, another company, or school. To do this, we must first determine the areas for comparison such as math scores, speed to market, market share, or manufacturing waste.
- Felt needs express a gap between current performance or skill level and the desired one. Felt needs are not necessarily performance needs and analysis should be able to detect this.
- Expressed needs are felt needs turned into actionable evidence such as waiting lists for certain programs or courses, indicating the need for additional resources.
- Anticipated or future needs identify changes that will occur in the future. New systems scheduled for installation, price instability, increased competition, and new manufacturing methods are examples.
- Critical incident needs are failures that are real and have significant consequences, such as chemical spills, natural disasters, and equipment failures. Critical needs are frequently identified after the fact, but can be anticipated by analyzing potential problems and asking "what if" questions. Without changes, what past incidents may happen again?
Approaches for gathering data
The internal or external client's request for training should be considered the presenting problem (Block, 2011). This will often be revealed as a symptom of the real problem, so you will need to advise additional information gathering even when the manager is sure he or she has identified the true problem. If the problem does turn out to be a training problem, you will be identifying the focus for training.
The more sources of information you can use, the more likely you are to arrive at accurate conclusions. In addition to expressing their views and ideas, many sources will also tell you what education and training is needed. Rather than accept or reject these ideas, use them as one piece of information to be later consolidated with others. Needless to say, you will be taking notes, recording your impressions, and jotting down unanswered questions you have. Experts also caution against analysis paralysis (Rossett, 2009), the quest for everything that might be related to the business need resulting in an effort that takes too long for the client and problem to be addressed in a timely manner. It's a good idea to establish a deadline for data gathering, typically two to three weeks, and another for analysis.
Interviewing. Gather the views and ideas from a number of sources from all relevant units and levels of the organization. Interviews can be conducted in-person, individually or in groups, by telephone, and teleconference. If workers are the intended targets of training, be sure to interview at least a few. When doing so, ask them questions to help identify master performers, those others go to for advice and problem solving. Spend time with these people plus anyone affected by their work such as supervisors and subordinates.
Surveys and questionnaires. Use these methods when you need to collect the views and ideas of a large number of people
Observation. It is helpful to observe processes and performers during regular business hours, including meetings and equipment. A watchful eye will often pick up on issues and problems not seen by those familiar with the environment and work. Idiosyncratic behavior by individuals necessitates the need to observe multiple performers, preferably at different times of day and days of the week. Whenever possible, you will also want to observe a range of performers, from expert to under-performers.
Existing data analysis. Production, sales, accident reports, meeting minutes and other data can yield important information, especially in identifying history and trends. Don't forget to review documents and correspondence, including e-mail, any time they are involved in the business process.
SME Analysis. When introducing new methods and technologies, the other approaches may not fully identify needs. In these cases, we must rely on the analysis and advice of subject matter experts with previous experience. Of course, this will not uncover every need in the current circumstance, but will help fill gaps unanswered by other approaches. Types of analysis include concept analysis, hierarchical analysis, and procedural task analysis. SMEs may be prone to developing their own training plans rather than describing the constituent parts and their interrelationships that you seek. It is your job to ensure, by clearly describing your needs, asking direct and probing questions, to obtain the information you seek.
Research Literature. Are there relevant trends, barriers, emergent issues, skills or technology in the industry? Does the research tell us what others have done? Especially when you are not informed of the industry, a literature search can educate you on the issues confronted by the industry.
Questions to address
Questions must be customized for the circumstance; however, important issues can be uncovered using questions like the following (Rossett, 2009):
- What is the problem? Why address it now and not some other time?
- What are the most costly errors being made? Which issues are the most critical?
- Why are employees having these problems? Whats getting in the way?
- What have you done to address it so far? What have been the results?
- How do your perceptions match what Ive described to you?
- If you ruled the organization, what would you do?
- What worries you?
- What are you doing that strikes others as best practices?
Various instruments, such as the force field analysis work sheets authored by Chevalier (2008), can be very useful for gathering information in a more structured manner. SWAT analysis (strengths, weaknesses, opportunities, threats) is a popular business strategy tool, and there are many more. These tools can be complex and are generally best administered by leading small groups through the process.
Consolidating and reporting your findings
Peter Block (2011), a highly regarded business consultant, recommends a time-tested and simple approach to consolidating and reporting your findings.
- Read through all the notes you have compiled while information gathering; do this in one setting. If your notes are very extensive, you may first need to summarize each set of notes from the different sources, and then look at your summaries all together.
- On a blank sheet of paper, list the issues that impress you as most important. Limit yourself to four or five items. What stands out for you should be what you want to stand out for your client.
- Use this list to focus and organize your report, prioritizing the items according to their impact. "Of all the things we might address, which would have the most benefit?" Other issues can be addressed in an appendix.
- Use language that is descriptive, focused, specific, brief, and simple. Avoid language that is judgmental, global, stereotyped, lengthy, or complicated.
If at all possible, meet with your client to present your findings and recommendations. Just as you have kept your report simple and straightforward, you want the presentation to follow the same plan. Block recommends the following agenda.
- Problem statement
- Why the problem exists (sources of the problem)
- What happens if the problem is not fixed (short-term and long-term)
- Recommended solutions
- Expected benefits
- Ask for client reactions
Depending on your expertise, you may want to focus on those solutions you can implement, plus how your actions will be affected by the issues you cannot address. We all know that employee training cannot solve problems caused by a toxic manager. Ignoring these other issues may get you the job, but would constitute a form of professional malpractice.
Task analysis
Designing instruction for performance issues, task analysis is considered the most critical part of the instructional design process (Morrison et al., 2011). Task analysis accomplishes three things:
- It defines the content required to resolve the performance problem.
- It forces the subject matter expert to work through each individual step, subtle steps become easier to define, and implicit knowledge surfaces.
- The instructional designer is able to view the content from the learner's perspective, gaining insight into appropriate instructional strategies.
An understanding of the learner's knowledge and background related to the task, obtained through the needs analysis, helps determine the beginning point for analysis as well as the depth and breadth of analysis. The instructional designer's role is to help and prod the subject matter expert to elaborate the task in a logical matter, and to ensure the descriptions are complete. The output of task analysis is documentation of the content to include in instruction.
Task analysis techniques
Morrison et al. (2011) suggest three specific task analysis techniques:
Topic or content analysis used to define the facts, concepts, principles, and rules that will make up the instruction. Content analysis identifies the content to be focused on, and the structure of the instructional components. Content analysis is described below in detail.
Procedural analysis examines tasks to identify the steps required to complete them. They are both observable (behavioral) and cognitive (information processing). Observable steps are best identified by observing and questioning the subject matter expert, identified as a master performer in this context. Cognitive steps are best uncovered by asking the SME to think aloud as he or she performs the task. Once outlined or charted by the instructional designer, the SME reviews it and fills in the gaps. Each step should be annotated with hints, cues, considerations, typical problems, etc. that are useful in its completion.
Task steps can be classified in at least three ways:
- by frequency of execution (frequently, occasionally, rarely)
- by order of difficulty (easy, moderate, difficult)
- gross to fine required motor skills
Complex cognitive tasks require additional information for instruction to be complete. These approaches ask the SME to articulate this additional information.
GOMS: Goals, Operators, Methods, and Selection analysis developed by Card, Moran & Newell (1983)
- Goals - what outcome(s) are expected from task completion
- Operators - actions required to accomplish the goals
- Methods - techniques, approaches, how-to's
- Selection rules - when to select one method over another
GOMS analysis is relatively quick and easy and accomplished with little cost, but fails to consider human factors such as errors, fatigue, the presence of others, etc.
ACTA: Applied cognitive task analysis developed by Militello & Hutton (2002)
- Identify and diagram 3-6 broad steps lacking in detail.
- Conduct a knowledge audit to generate examples for each broad step, indicating sequence and the cognitive skills required for each (diagnosis and prediction, situation awareness, perceptual skills, tricks of the trade, improvising, recognizing anomalies, compensating for equipment limitations, judgment, assessment, problem-solving). Questions to uncover these skills:
- Past & Future: Is there a time when you walked into the middle of a situation and knew exactly how things got there and where they were headed?
- Big Picture: Can you give me an example of what is important about the "big picture" for this task?
- Noticing: Have you had experiences where part of a situation just popped out at you; where you noticed things going on that others didnt catch? What is an example?
- Job Smarts: When you do this task, are there ways of working smart or accomplishing more with less that you have found especially useful?
- Self-Monitoring: Can you think of a time when you realized that you would need to change the way you were working in order to get the job done?
- Conduct a simulation interview by identifying and showing the expert a challenging scenario and then probing their assessment, actions, critical cues, and potential errors.
- Construct a cognitive demands table to consolidate and synthesize the data, identifying each difficult cognitive element, the reason it is difficult, common errors of execution, and issues and strategies used to solve the problem. An example for search and rescue operations:
| Table 2. Sample Cognitive Demands table | |||
|---|---|---|---|
| Difficult Cognitive Element | Why difficult? | Common errors | Cues and Strategies used |
| Knowing where to search after an explosion | Novices may not be trained in explosions. Other training suggests to start at the source. Not everyone knows about MSDS. | Novice would be likely to start at source of explosion. Starting at source is a rule of thumb. | Start where likely to find victims. Refer to MSDS. Consider type of structure. |
| Finding victims in a burning building | There are lots of distracting noises. Own breathing distracts. | Novices think their own breathing is a victim. | You and partner stop, hold breath, and listen for crying, talking, bumping into things. |
Content analysis
"Instruction consists of two major components: what to teach, or subject-matter content, and how to teach, or instructional strategies. There is a difference between knowledge - what we know - and skill - what we do with what we know. Performance requires both knowledge and skill" (Merrill, 2013). Content analysis addresses the subject-matter content. In addition to being a method for task analysis, content analysis is also useful for a variety of other situations.
Content analysis is a technique for understanding and summarizing a collection of text. An example of this comes from summarizing needs analysis documents, where we look for common themes to guide our conclusions and recommendations. For larger content collections, analysis is typically accomplished in layers, such as major categories, then subcategories, then details (Morrison et al., 2011). This analysis is accomplished through both qualitative and quantitative means. Peter Block's approach (2011) of reading through all documentation collected during needs analysis and then writing down the most important common themes is an example of qualitative analysis. This documentation can also be examined quantitatively by calculating the count of all the words within the documents. "Word cloud" websites perform this analysis when you copy and paste text, resulting in a collection of terms with higher frequency counts displaying in larger font size. In serious matters, the analysis would be shown through spreadsheets, statistics, and graphs. Here's an example using this web page in Wordle (http://www.wordle.net/).
|
|---|
| 2. Quantitative content analysis with Wordle |
Uses of content analysis
The needs analysis example above is just one of many uses for content analysis. A few more examples:
- Separating out must-know from nice to know or supplemental information.
- Deciding what topics should be covered within a course. Note that this is a common strategy for textbook publishers. This does not preclude instructors from conducting their own analysis to determine whether or not to use all the publishers materials, or to add their own.
- Separating out just-in-time instruction from supporting information in the 4C/ID design model (van Merrriënboer, et al., 2002).
- Determining the content domain (cognitive, psychomotor, affective, interpersonal, perceptual) in order to shape the best instructional strategies.
- Determining content level (module, lesson, topic) in order to properly order and group content.
- Determining the most common problems encountered by learners as they complete instruction based on the questions they ask.
- Examining student discussion threads to identify common beliefs, issues, controversies, etc.
Of the examples above, a content domain analysis is one of the more fruitful uses of content analysis for instructional designers. As the designer analyzes course content, a blueprint of the course emerges, outlining the overall structure and sequence of learning (Chyung & Treñas, 2009).
Types of Knowledge
Though discussed earlier in the Language of learning, it is useful here to revisit the types of knowledge we can use to further classify content. Merrill (2013) discusses four type of knowledge:
Verbal/declarative knowledge: information about something (facts associated with a single entity, activity, or process that cannot be generalized), and being a part of an entity, activity, or process (name, description, relation to the whole, location within the whole). Not generally a learning goal in itself, but supportive of other kinds of learning.
Conceptual knowledge: definition, list of defining properties, examples (illustration, representation, vivid description, or image of a specific event, person, or thing) and non examples (incomplete examples, pseudo/near examples). Learning to recognize instances from a class of objects, symbols, or events. Characterized by a common set of properties; distinguishable from each other, yet sharing common properties. Also, not generally the goal of learning, but supportive of other kinds.
Procedural knowledge: how-to, steps and sequence of actions, examples and non examples. A way for learners to act on their environment. An object or situation is modified by the sequential series of actions. Appropriate when the content specifies a sequence of activities for the learner to do or accomplish a goal or consequence. Often a primary goal of instruction.
Process knowledge: conditions and consequences, examples and non examples. Enables learners to comprehend or modify their environment. Often stated in if-then terms; if these conditions (properties of a situation) are changed, then these consequences (change in conditions) follow. Conditions can change as a natural occurrence or caused by some action on the part of the learner.
Qualities of knowledge
Jong (1996) describes four qualities of knowledge that can also be useful in categorizing content.
| Table 3. Qualities of Knowledge | ||
|---|---|---|
| Quality | From . . . | To . . . |
| Level | Surface, superficial, rote, and concrete | Deep expertise associated with comprehension, abstraction, and application. Characterized by critical evaluation and judgment. Thoroughly processed and stored in memory. |
| Structure | Isolated and random | Highly structured and chunked into large meaningful units and hierarchies |
| Automation | Explicit, conscious and step-wise | Subconscious and intuitive, tacit, expert |
| Modality | verbal, visual, tactile, gustatory, olfactory, or a combination | |
| Generality | General and widely applicable (e.g., problem-solving methodology, math techniques) | Domain specific; tailored to the domain and subdomain, situational (e.g., solving calculus problems, calculating the raw materials necessary for a building project) |
Summary
- Front-end analysis is the surest way to ensure that instruction is necessary to address presented needs, problems, and opportunities.
- Needs analysis focuses on the organization and its performance needs. Summarizing and reporting the needs analysis is integral to the process.
- Task analysis defines the instructional content of individual and group performance issues.
- Content analysis is also used to define the instructional content of training and education issues.
- Multiple methods for conducting the analyses were discussed.
⇑ ⇑ ⇑ Up to Course development
⇑ ⇑ ⇑ ⇑ Up to Home