Resources
 

White Papers

Cornerstone is professionally committed to creating effective, research-based solutions to our clients' need to identify and prepare top talent to become effective leaders and managers. Our system is based on peer-reviewed, published research and on basic statistical principles universally recognized as tools for studying and evaluating this type of material. This approach protects organizations from investing in programs that appear effective but do not pass the test of time.

To learn more about why we do what we do and how we are different from (and more effective than) traditional assessment centers, click here to request a copy of our white papers:

  • Much, Much More Than Your Father's Assessment Center
  • Why We Are Different

Memberships

Cornerstone maintains current memberships with the following organizations:

We are also a Preferred Industry Partner (PIP) of SPBT: The Society of Pharmaceutical & Biotech Trainers (www.spbt.org)

Confused about Our “Jargon”? This Glossary Will Help.

We believe an educated client is a happy client. That’s why we’ve included this list of words and phrases used throughout the Cornerstone site. And if you need more detail than these definitions provide, please feel free to call us at any time.

Achievement vs Aptitude

Measures of achievement assess what one has learned or the skills they have developed. Measures of aptitude assess concepts like intellectual ability and personality traits, and address someone's potential to develop and learn. As an analogy, the Law School Admissions Test (LSAT) is an aptitude test designed to predict success in law school. In contrast, a state bar exam is an achievement measure that evaluates what was learned in law school and whether perspective lawyers possess the basic knowledge and skills necessary to practice law. Law firms hiring young lawyers are far more interested in whether candidates passed the state bar exam than their LSAT scores.

Cornerstone is committed to the appropriate use of aptitude measures. Thus, we have developed Scenarios® as a high utility measure of the aptitude to take full advantage of the developmental experiences offered to candidates prior to attendance at a Job Function Simulation (JFS) program. Scenarios® assists clients in:

  • Identifying potential candidates who may have been overlooked by management
  • Eliminating potential candidates who would not—due to issues of ability and/or motivation—be successful in the leadership development program.

Back to top

Adverse Impact

Adverse impact deals with the issue of discrimination (intentional or not) against groups of citizens protected under the Civil Rights Acts of 1964 and 1991. By law, a selection or promotion procedure cannot treat members of protected classes (e.g., racial minorities) unfairly. According to the Equal Employment Opportunity Commission (EEOC), a selection procedure has adverse impact if its use results in the minority group being selected at a rate less than 80% of that of the majority group a requirement known as the Four-Fifths Rule.

For example, if an organization utilizes an assessment center program that results in 50% of white candidates being promoted, but only 30% of black candidates being promoted, this is direct evidence of adverse impact for the assessment center (i.e., 30% minority selection is only 60% of the majority ratio, as .30/.50=.60). If an organization using a selection procedure with adverse impact is challenged in court, the organization may face severe financial penalities.

Some organizations seem to believe that if they do not use systematic procedures or if they use a series or combination of selection tools they can somehow cloud the selection procedure in a way that will allow them to get around employment law. In truth, these organizations probably increase their jeopardy. Adverse impact is adverse impact regardless of how it is generated.

Back to top

Aptitude

Measures of achievement assess what one has learned or the skills they have developed. Measures of aptitude assess concepts like intellectual ability and personality traits, and address someone's potential to develop and learn. As an analogy, the Law School Admissions Test (LSAT) is an aptitude test designed to predict success in law school. In contrast, a state bar exam is an achievement measure that evaluates what was learned in law school and whether or not perspective lawyers possess the necessary basic knowledge and skills to practice law. Law firms hiring young lawyers are far more interested in whether or not candidates passed the state bar exam than their LSAT scores.

Cornerstone is committed to the appropriate use of aptitude measures. Thus, we have developed the Management Potential Review (MPR) and Scenarios-DMtm as high utility measures of the aptitude to take full advantage of the developmental experiences offered to candidates prior to attendance at a Job Function Simulation (JFS) program. The MPR and Scenarios-DMtm assist clients in (1) identifying potential candidates who may have been overlooked by management, and (2) eliminating potential candidates who would not - due to issues of ability or motivation - be successful in the management development program.

Back to top

Assessment Center

The assessment center is not a physical place, but is performance-based procedure that is designed to measure managerial aptitude or potential. Assessment centers first were used in industry in the 1950s and are structured around the measurement of managerial dimensions or competencies. Although scientific research has indicated that assessment centers possess reasonable predictive validity for management jobs, they are problematic for a number of reasons. First, assessment centers are often marketed as "off the shelf" or "cookie cutter" programs by large consulting firms trying to mass-produce their programs. In these programs, dimensions may not be relevant (e.g., job-specific), and the exercises used to generate behavior may not have content validity. Second, because the evaluation of assessment center performance is dimension-based, assessment centers are limited in their ability to provide meaningful developmental feedback for candidates. Rather than addressing managerial skills that can usually be developed, dimension-driven feedback frequently is structured around personality traits or aptitudes, which are highly difficult for people to change. In addition, scientific research conducted over the last 20 years indicated that dimension/competency evaluations that are used for developmental diagnosis and planning are seriously flawed. (See exercise effect.) Finally, when used to predict performance, assessment centers have poor utility. That is, the return on investment for these programs is low because of the high cost of program administration and the modest (if any) improvement in predictive validity over much less expensive paper-and-pencil (or website-administered) predictors such as situational judgment tests (SJTs).

Cornerstone's Job Function Simulation (JFS) programs were designed to capitalize on some positive aspects of assessment centers while avoiding the critical pitfalls. Because JFS programs are job-specific and task-based, there is no need to evaluate personality-derived dimensions. Our programs also result in job-specific and actionable (rather than general, vague and trait-based) developmental feedback.

Back to top

Content Validity

Content validity is the degree to which a test or some other selection procedure adequately captures critical task-based elements of the job for which it is being used. Developing a test or procedure from a well-designed job analysis can enhance content validity. In turn, establishing the content validity of a selection procedure improves its legal defensibility. All components of Cornerstone's managerial selection and development programs are created from job analysis findings and possess content validity.

Back to top

Dimensions (in Assessment Centers)

Dimensions are personal characteristics or KSAs (knowledge, skills, ability) thought to be related to performance of a job. The nature of these dimensions will vary between different assessment center providers, but common examples include such constructs as “decisiveness,” “initiative,” “stress tolerance” and “interpersonal sensitivity,” to name a few. Assessment centers attempt to measure these dimensions through multiple activities or exercises. Evaluations of leadership aptitude or potential—as well as developmental feedback—are structured around these dimensional evaluations during the assessment center.

Assessment center providers may attempt to frame these dimensions as aspects of the job, rather than characteristics of the candidate. However, this merely confuses the measurement issue at hand. Many dimensional constructs (e.g., “initiative” and “decisiveness”) can only be thought of as personal characteristics—as opposed to critical job functions.

The use of dimensions in traditional assessment centers is problematic for several reasons.

  • The dimensions themselves are often ill-defined and vague, and dimensional definitions will vary from setting to setting. Even when assessment center providers assign “behavioral anchors” to each dimension, these designations are largely arbitrary. Even trained evaluators will differ in their interpretations of these “behavioral” definitions.
  • Behavioral research conducted in university settings (by persons who have no financial interest in the outcomes of the research) has shown that dimensional evaluations cannot be reliably extracted from assessment center exercises. So, the level of “stress tolerance” displayed by a candidate during a simulated group meeting is largely independent of the “stress tolerance” exhibited by the same candidate during a challenging one-on-one interpersonal interaction. This is problematic if exercises are regarded as multiple opportunities to evaluate candidates’ standing on various dimensions.
  • Dimensions may not be related to success at the job in question. Frequently, assessment center providers will “force fit” their dimensions to meet written job descriptions—or worse—do so in a post hoc fashion. Thus, performance at the assessment center may offer a very poor measure of whether a candidate is ready for promotion to a position.
  • Because feedback is structured around personal characteristics (i.e., dimensions) as opposed to job requirements, assessment center developmental feedback may have little value to participants. Even if valid, dimensional evaluations may relate to personality characteristics (e.g., “stress tolerance”) that are very difficult to change in adults. In fact, independent research has shown that candidates react more favorably—and are more motivated to develop their skills—when developmental feedback is focused on tasks and job responsibilities rather than abstract, vague “dimensions.”

Cornerstone’s Job Function Simulation (JFS) programs avoid the pitfalls of dimensional assessment by focusing evaluations and developmental feedback on job responsibilities as opposed to abstract personal characteristics. In relation to the traditional assessment center, the JFS process is more straightforward, more job-relevant and easier to defend if challenged.

Equal Employment Opportunity Commission

The EEOC Uniform Guidelines on Employee Selection Procedures outline the legal requirements and constraints for personnel selection and promotion programs and decisions in the United States. Of particular interest to the EEOC guidelines is ensuring that a selection and/or promotion system does not result in adverse impact against groups of citizens protected under the Civil Rights Acts of 1964 and 1991. The Uniform Guidelines may be viewed at the following address:

www.uniformguidelines.com/uniformguidelines.html

Back to top

Exercise Effect

If the dimensions that are traditionally measured in assessment centers exist as meaningful entities, correlations of measures of the same dimensions measured across different sources ("exercises") should be higher than correlations with measures of different dimensions. In other words, the intercorrelations of measures of the same construct taken in different settings should be higher than correlations of those measures with logically unrelated variables. If dimensions exist in a meaningful manner, they should be expected to "hold together". As it turns out, there are now over 25 published peer reviewed studies that document higher correlations between evaluations of different dimensions evaluated within the same "exercise" than evaluations of the same dimensions evaluated across different exercises. Dimension evaluations correlate higher with things they are supposed to be independent of than with themselves.

This phenomenon was first called "exercise effect" in 1982. To professionals specially trained in the development of human performance measures this is a very major problem that undermines the foundation of traditional assessment centers. Job Function Simulation programs (which do not evaluate performance using the dimension paradigm) are designed to take advantage of the exercise effect phenomenon and have been recognized as a solution to this problem.

Back to top

High Fidelity Work Sample

A high-fidelity work sample is a very realistic, near-true-to-life job simulation. In a high fidelity work sample, the candidate's effectiveness when handling essential functions relevant to a specific job is evaluated. The content of a high fidelity work sample should always be created from the findings of a job analysis. The focus should be on actual problems and responsibilities incumbents actually encounter and solve. The traditional assessment center method of identifying the "dimensions associated with job success" and generating generic or near-generic "exercises" that supposedly provide behavior relevant to the dimensions does not result in a high fidelity job sample.

In contrast, Cornerstone’s Job Function Simulation (JFS) programs are based on highly detailed industry and organization-specific job analyses of the District Sales Manager position. Scientific research in academic and organizational settings has indicated that high fidelity work samples possess strong predictive validity relative to other selection methods. This is a very robust finding and applies across all occupational groups.

Back to top

Job Analysis

Job analysis studies provide the foundation for the design of different HR interventions. A job analysis that is to be used to design a training program could identify training needs or things that effective managers do to solve problems. A job analysis that is to be used to develop a predictive test would usually identify the aptitudes or traits associated with job success.

The job analyses that Cornerstone conducts are designed to provide the understanding and insight necessary to design work sample simulations that possess unusually high levels of content validity. Cornerstone's job analysis procedures are organization-specific and highly task-based. The primary analysis instrument is an extensive questionnaire that is developed from structured interviews with current District Sales Managers and their supervisors (Regional Sales Directors). These questionnaires are composed of tasks, situations and problems that managers must handle. Ratings are developed for both the occurrence and frequency of each situation and the importance of handling it effectively. The primary means of addressing each situation (e.g., face-to-face meeting, voicemail, telephone, written correspondence, etc.) is also identified. Relevant (frequently occurring and important) problems are positioned in simulations that allow participants to respond in ways similar to how they would on the job.

An appropriate job analysis is critical to establishing the legal defensibility of any procedure used in a selection process. Because they provide for the creation of "audit trails" between job and simulation content (every problem confronted and situation faced in the simulation can be directly associated with one or more questionnaire items) Cornerstone programs not only assure program quality and relevance but possess an unusually high level of content validity that enhances their legal defensibility.

Back to top

Predictive Validity

Predictive validity refers to the ability of a test or some other selection procedure to forecast future performance. Predictive validity is estimated by the degree of statistical correlation between test performance and job performance. All elements of Cornerstone's management development program that involve selection (e.g., Scenarios® and each Job Function Simulation) are based on techniques shown through scientific research to possess predictive validity, because the content of these measures reflects the content of the District Sales Manager position.

Back to top

Rating Biases

Rating biases are systematic sources of inaccuracy that impact the quality of all types of performance evaluations. They have nothing to do with actual performance and are a major source of error in many commonly used evaluation procedures. Some common source of rating bias include the following

  • Halo effect. Halo effect occurs when a rater's general impression of a candidate clouds the rater's ability to make accurate evaluations of specific skill or performance issues. A "top-down" impression of the candidate is formed, which colors all subsequent evaluations.
  • Scale restriction effect. Scale restriction effect occurs when a rater is overly "hard" or "lenient", and either systematically inflates or depresses all evaluations.
  • Central tendency effect. The central tendency effect occurs when raters are systematically hesitant to assign very good or very poor evaluations of performance, and cluster most of their ratings around "average.”
  • "Similar-to-me" effect. Similar-to-me effect occurs when a rater assigns a candidate favorable evaluations as a function of the perception that the candidate is similar to the rater in terms of characteristics such as race, sex, dress, attitudes, personality, or values.
  • Contrast effect. The contrast effect is a function of the ordering of candidates during an evaluation process. A given candidate's evaluation is influenced by the evaluation of a previous candidate. If a candidate performs at a satisfactory level, but follows a candidate who has performed extremely well (e.g., is a "tough act to follow"), the satisfactory candidate may receive an evaluation than is less than what is deserved. Similarly, that same “satisfactory” candidate may receive a much more favorable evaluation than is deserved if he or she follows someone who has done poorly.
  • Stereotype effect. The stereotype effect occurs when a person's performance is evaluated (at least partially) as a function of the rater's preconceived notions about the impact that membership in a recognizable group or social category has on performance. Stereotyping may occur as a function of candidate sex, race, ethnicity, age, religious affiliation, and so forth.

The design of Cornerstone's Job Function Simulation program minimizes each of these rating biases by:

  • Ensuring that each candidate is evaluated by an evaluation team member in one and only one simulation
  • Training managers assigned to evaluation teams in sources of rating bias and how to avoid them
  • Utilizing only job-content experts (typically current or former District Sales Managers and Regional Sales Directors) as evaluation team members
  • Using an objective, behavior-based method of evaluating JFS simulation element performance
    Utilizing a consensus-based evaluation process
  • Drawing upon a culturally diverse pool of evaluation team members
  • Avoiding a dimension- and inference-based evaluation structure

Back to top

Readiness

Readiness is simply the extent to which a candidate representative is prepared for promotion to District Sales Manager. This is very different than an estimate of a candidate's aptitude to become an effective district manager. Readiness estimates indicate how well people have used their aptitudes to develop managerial skills and address current capabilities, instead of an estimate of what might occur in the future. Job Function Simulation programs result in an estimate of each candidate's readiness for promotion. If a candidate's performance indicates that he or she is not currently ready for promotion, developmental recommendations associated with necessary skill development are provided.

Back to top

Realistic Job Preview

Realistic job previews (RJPs) are meaningful presentations of most cogent favorable and unfavorable elements of a job to people who have little reason to fully understand and appreciate the content of a given job. RJPs allow people to make informed choices about interests, motivations and abilities to perform important job tasks and responsibilities. Written and verbal job descriptions are seldom sufficiently realistic to achieve these goals. Research conducted in both academic and organizational settings has consistently demonstrated that the use of RJPs enhances organizational performance by lowering turnover costs. In addition, it is only logical to assume that when someone is unpleasantly surprised about some major aspect of a new job it will be very difficult for the person to maintain the interest and motivation necessary to stimulate effective performance.

Cornerstone’s intraspec® program functions as an effective RJP in that it presents the candidate with many important aspects of the district manager job. Since intraspec® is positioned early in the developmental system, it allows candidates to remove themselves from the process before they have committed significant personal or company resources to an inappropriate career move. In ways, all Cornerstone system components function as RJPs. leadership ASCENTials™ requires at least vicarious involvement in many District Sales Manager responsibilities, and Job Function Simulations result in an immersion in many critical aspects of the DSM position.

Back to top

SIOP Principles for the Validation and Use of
Personnel Selection Procedures

Of interest to anyone involved in employee selection and staffing are the Society for Industrial and Organizational Psychology (SIOP) Principles for the Validation and Use of Personnel Selection Procedures (AKA “The Principles”). Although not binding in any legal sense, the Principles offer professional and ethical recommendations for the development, validation and utilization of selection procedures. The Principles were developed by a panel of industrial psychologists and selection experts working in both academic and organizational settings.

All components of Cornerstone's management development and selection programs were developed to be consistent with the Principles. The Principles may be viewed and downloaded at the SIOP website, or by clicking on the following link:

http://www.siop.org/_Principles/principlesdefault.htm

Back to top

Situational Judgment Test (SJT)

SJTs are scenario-based written tests that evaluate candidate/applicant skills solving problems similar to those they would handle in the target position. SJTs present respondents with a series of job-related problems and/or scenarios and require them to select from several options (decisions, tactics, etc.) that describe possible solutions to these situations. Cornerstone's Scenarios® measure the quality of managerial decision making for the District Sales Manager position. When SJT content is developed from job analysis findings, SJTs can predict future managerial job performance about as well as assessment centers at a fraction of the cost and inconvenience. Cornerstone recommends that Scenarios® be used in conjunction with other company information in identifying candidates with potential to realize maximum benefits of intraspec®, leadership ASCENTIALS™ and the Job Function Simulation.

Back to top

Society for Industrial and Organizational Psychology

SIOP (the Society for Industrial and Organizational Psychology) is a professional society composed of specialists in industrial-organizational psychology, human resource management and related fields. Members may be scientists (academicians), practitioners (usually consultants or employees of large-scale organizations or government agencies) or students pursuing a degree in industrial-organizational psychology or related fields. More information on SIOP may be obtained at:

www.siop.org

Back to top

Utility

Utility is a quantified means of estimating return on investment of a selection or training procedure. Utility is estimated by the financial gains resulting from the use of a procedure, less the costs of the procedure. Utility is a function of several variables, most importantly

  • The degree of predictive validity of the selection method under consideration,
  • How selective the organization chooses to be in the hiring or promotion process,
  • The dollar value of effective versus ineffective performance for the job in question, and
  • The cost of the selection procedure itself.

Utility can be estimated for a training intervention through a quantitative estimate of the dollar value of improved job performance resulting from the training intervention, combined with cost.

Utility should be a crucial consideration for an organization in its decision to adopt a particular selection procedure. However, many decision makers in organizations are unaware of the existence of the utility method for estimating ROI.

Utility for a valid personnel selection system is typically much greater than assumed by management, and valid selection systems almost always pay for themselves very quickly. Upon request, Cornerstone will happily compute organization-specific utility estimates for our programs.

Back to top