Loading...
HomeMy WebLinkAbout16-10 Resolution No. 16-10 RESOLUTION AUTHORIZING EXECUTION OF AN AGREEMENT WITH INDUSTRIAL/ORGANIZATIONAL SOLUTIONS, INC. FOR POLICE SERGEANT EXAM TESTING SERVICES IN CONNECTION WITH THE DESIGN OF POLICE WRITTEN EXAM AND ASSESSMENT CENTER PROJECT BE IT RESOLVED BY THE CITY COUNCIL OF THE CITY OF ELGIN, ILLINOIS, that Sean R. Stegall, City Manager, be and is hereby authorized and directed to execute an Agreement on behalf of the City of Elgin with Industrial/Organizational Solutions, Inc. for police sergeant exam testing services in connection with the design of police written exam and assessment center project, a copy of which is attached hereto and made a part hereof by reference. s/David J. Kaptain David J. Kaptain, Mayor Presented: February 10, 2016 Adopted: February 10, 2016 Omnibus Vote: Yeas: 9 Nays: 0 Attest: s/Kimberly Dewis Kimberly Dewis, City Clerk AGREEMENT THIS AGREEMENT made and entered into this 10 day of February, 2016, by and between the CITY OF ELGIN, an Illinois municipal corporation (hereinafter referred to as "CITY") and_Industrial Organizational Solutions, Inc. (hereinafter referred to as"CONSULTANT'). WHEREAS, the CITY desires to engage the CONSULTANT to furnish certain professional services in connection with the pesian of Police Written Exam and Assessment Center(hereinafter referred to as the PROJECT). AND WHEREAS,the CITY desires to engage the CONSULTANT to furnish certain goods and professional services for RFP Number: 15-050 for: Design of Police Promotional Assessment Center(hereinafter referred to as the PROJECT);and WHERAS, the CONSULTANT represents that he has the necessary expertise and experience to furnish such goods and services upon the terms and conditions set forth herein below. NOW, THEREFORE, it is hereby agreed by and between the CITY and the CONSULTANT that the CITY does hereby retain the CONSULTANT for and in consideration of the mutual promises and covenants contained herein, the sufficiency of which is hereby acknowledged to act for and represent it in matters involved in the PROJECT as described herein, subject to the following terms and conditions and stipulations,to-wit: 1. SCOPE OF SERVICES, a. All work hereunder shall be performed under the direction of the Police Chief and the Human Resources Director of the CITY, herein after referred to as the "PROJECT MANAGERS". b. Conducting and implementing the tasks for Design of Police Promotional Assessment Center as required by RFP number 15-050. c. A detailed Scope of Services is attached hereto as Attachment A. 2. PROGRESS REPORTS a. An outline project milestone schedule is attached hereto as Attachment B. b. The CONSULTANT will submit to the PROJECT MANAGERS a status report keyed to the project schedule. A brief narrative will be provided identifying progress, findings and outstanding issues. 3. WORK PRODUCTS All work products prepared by the CONSULTANT pursuant hereto including, but not limited to, reports, designs, calculations, work drawings, studies, photographs, models and recommendations shall be the property of the CITY and shall be delivered to the CITY upon request of the PROJECT MANAGERS provided, however,that the CONSULTANT may retain copies of such work products for its records. CONSULTANT's execution of this Agreement shall constitute CONSULTANT's conveyance and assignment of all right, title and interest, including but not limited to any copyright interest by the CONSULTANT to the CITY of all such work product prepared by the CONSULTANT pursuant to this Agreement. The CITY shall have the right either on its own or through other CONSULTANTs as determined by the CITY to utilize and/or amend such work product. Any such amendment to such work product shall be at the sole risk of the CITY. Such work product is not intended or represented to be suitable for reuse by the CITY on any extension to the PROJECT or on any other project,and such reuse shall be at the sole risk of the CITY without liability or legal exposure to the CONSULTANT. 1 4. PAYMENTS TO THE CONSULTANT(Not To Exceed Method) A. The City shall reimburse the CONSULTANT for goods and professional services described in Attachment A.For goods and services provided the CONSULTANT shall be paid a total fee not to exceed 6_25630.00_regardless of the actual costs incurred by the CONSULTANT unless substantial modifications to the scope of the work are authorized in writing by PROJECT MANAGERS. B. The CITY shall make periodic payments to the CONSULTANT based upon actual progress within 30 days after receipt and approval of invoice. Said periodic payments to the CONSULTANT shall not exceed the amounts shown above. 5. INVOICES A. A The CONSULTANT shall submit invoices in a format approved by the CITY. Progress reports (2B above)will be included with all payment requests. B. The CONSULTANT shall maintain records showing actual time devoted and cost incurred. The CONSULTANT shall permit the authorized representative of the CITY to inspect and audit all data and records of the CONSULTANT for work done under this Agreement.The CONSULTANT shall make these records available at reasonable times during the Agreement period, and for a year after termination of this Agreement. 6. TERMINATION OF AGREEMENT Notwithstanding any other provision hereof,the CITY may terminate this Agreement at any time upon fifteen (15) days prior written notice to the CONSULTANT. In the event that this Agreement is so terminated, the CONSULTANT shall be paid for services actually performed and reimbursable expenses actually incurred prior to termination, except that reimbursement shall not exceed the task amounts set forth under Paragraph 4 above. 7. TERM This Agreement shall become effective as of the date the CONSULTANT is given a notice to proceed and, unless terminated for cause or pursuant to Article 6, shall be deemed concluded on the date the CITY determines that all of the CONSULTANTs work under this agreement is completed. A determination of completion shall not constitute a waiver of any rights or claims which the CITY may have or thereafter acquire with respect to any term or provision of the Agreement. 8. NOTICE OF CLAIM If the CONSULTANT wishes to make a claim for additional compensation as a result of action taken by the CITY,the CONSULTANT shall give written notice of his claim within 15 days after occurrence of such action. No claim for additional compensation shall be valid unless so made. Any changes in the CONSULTANTs fee shall be valid only to the extent that such changes are included in writing signed by the PROJECT MANAGERS and the CONSULTANT. Regardless of the decision of the PROJECT MANAGERS relative to a claim submitted by the CONSULTANT, all work required under this Agreement as determined by the PROJECT MANAGERS shall proceed without interruption. 9. BREACH OF CONTRACT If either party violates or breaches any term of this Agreement, such violation or breach shall be deemed to constitute a default, and the other party has the right to seek such administrative, contractual or legal remedies as may be suitable to the violation or breach; and, in addition, if either party, by reason of any default, fails within fifteen (15) days after notice thereof by the 2 other party to comply with the conditions of the Agreement, the other party may terminate this Agreement. Notwithstanding the foregoing or anything else to the contrary in this agreement,with the sole exception of the monies the CITY has agreed to pay to the CONSULTANT pursuant to Section 4 hereof, no action shall be commenced by the CONSULTANT or any other related entity against the CITY for monetary damages. CONSULTANT hereby further waives any and all claims or rights to interest on money claimed to be due pursuant to this Agreement, and waives any and all such rights to interest which it claims it may otherwise be entitled pursuant to law, including, but not limited to, the local government Prompt Payment Act(50 ILCS 501/1,et seq.),as amended,or the Illinois Interest Act (815 ILCS 205/1, et seq.), as amended. The parties hereto further agree that any action by the CONSULTANT arising out of this Agreement must be filed within one (1) year of the date the alleged cause of action arose or the same will be time barred. The provisions of this paragraph will survive any expiration, completion and/or termination of this Agreement. 10. INDEMNIFICATION To the fullest extent permitted by law, CONSULTANT agrees to and shall indemnify, defend and hold harmless the CITY, its officers,employees,agents, boards and commissions from and against any and all claims, suits, judgments, costs, attorney's fees, damages or other relief, including but not limited to workers compensation claims, in any way resulting from or arising out of negligent actions or omissions of the CONSULTANT in connection herewith, including negligence or omissions of employees or agents of the CONSULTANT arising out of the performance of this Agreement. In the event of any action against the CITY, its officers, employees, agents, boards or commissions, covered by the foregoing duty to indemnify, defend and hold harmless such action shall be defended by legal counsel of the CITY's choosing.The provisions of this paragraph shall survive any expiration, completion and/or termination of this Agreement. 11. NO PERSONAL LIABILITY No official, director, officer, agent or employee of the CITY shall be charged personally or held contractually liable under any term or provision of this Agreement or because of their execution,approval or attempted execution of this Agreement. 12. INSURANCE A.Comprehensive Liability.The CONSULTANT shall provide, pay for and maintain in effect, during the term of this Agreement,a policy of comprehensive general liability insurance with limits of at least $1,000,000 aggregate for bodily injury and $1,000,000 aggregate for property damage. The CONSULTANT shall deliver to the Purchasing Director a Certification of Insurance naming the CITY as additional insured. The Certificate of Insurance which shall include Contractual obligation assumed by the CONSULTANT under Article 10 entitled indemnification" shall be provided. The policy shall not be modified or terminated without thirty (30) days prior written notice to the DIRECTOR The Certificate of Insurance which shall include Contractual obligation assumed by the CONSULTANT under Article X entitled"Indemnification"shall be provided. This insurance shall apply as primary insurance with respect to any other insurance or self- insurance programs afforded to the CITY. There shall be no endorsement or modification of this insurance to make it excess over other available insurance, alternatively, if the insurance states that it is excess or prorated, it shall be endorsed to be primary with respect to the CITY. B. Comprehensive Automobile Liability. Comprehensive Automobile Liability Insurance 3 covering all owned, non-owned and hired motor vehicles with limits of not less than $500,000 per occurrence for damage to property. C. Combined Sinai.Limit Policy. The requirements for insurance coverage for the general liability and auto exposures may be met with a combined single limit of $1,000,000 per occurrence subject to a$1,000,000 aggregate. D. Professional Liability. The CONSULTANT shall carry Professional Liability Insurance covering claims resulting from error,omissions or negligent acts with a combined single limit of not less than$1,000,000 per occurrence.A Certificate of Insurance shall be submitted to the Purchasing Director as evidence of insurance protection. The policy shall not be modified or terminated without thirty (30) days prior written notice to the Purchasing Director. 13. QONSTRUCTION MEANS.METHODS.TECHNIQUES. SEQUENCES,PROCEDURES AND SAFETY The CONSULTANT shall not have control over or charge of and shall not be responsible for construction means, methods, techniques, sequences or procedures, or for safety precautions and programs in connection with the construction, unless specifically identified in the Scope of Services. 14.NONDISCRIMINATION In all hiring or employment made possible or resulting from this Agreement, there shall be no discrimination against any employee or applicant for employment because of sex, age, race, color, creed, national origin, marital status, of the presence of any sensory, mental or physical handicap, unless based upon a bona fide occupational qualification, and this requirement shall apply to, but not be limited to,the following: employment advertising, layoff or termination,rates of pay or other forms of compensation and selection for training,including apprenticeship. No person shall be denied or subjected to discrimination in receipt of the benefit of any services or activities made possible by or resulting from this Agreement on the grounds of sex, race, color,creed, national origin, age except minimum age and retirement provisions, marital status or the presence of any sensory, mental or physical handicap. Any violation of this provision shall be considered a violation of a material provision of this Agreement and shall be grounds for cancellation,termination or suspension, in whole or in part, of the Agreement by the CITY. 15.ASSIGNMENT AND SUCCESSORS, This Agreement and each and every portion thereof shall be binding upon the successors and the assigns of the parties hereto; provided, however,that no assignment shall be made without the prior written consent of the CITY. 16.DELEGATIONS AND SUBCONTRACTORS Any assignment, delegation or subcontracting shall be subject to all the terms, conditions and other provisions of this Agreement and the CONSULTANT shall remain liable to the CITY with respect to each and every item, condition and other provision hereof to the same extent that the CONSULTANT would have been obligated if it had done the work itself and no assignment, delegation or subcontract had been made. Any proposed subcontractor shall require the CITY's advanced written approval. 17.NO CO-PARTNERSHIP OR AGENCY This agreement shall not be construed so as to create a partnership, joint venture, employment or other agency relationship between the parties hereto. 4 18.SEVERABILITY The parties intend and agreed that, if any paragraph, sub-paragraph, phrase, clause or other provision of this Agreement, or any portion thereof, shall be held to be void or otherwise unenforceable,all other portions of this Agreement shall remain in full force and effect. 19.HEADINGS The headings of the several paragraphs of this Agreement are inserted only as a matter of convenience and for reference and in no way are they intended to define, limit or describe the scope of intent of any provision of this Agreement, nor shall they be construed to affect in any manner the terms and provisions hereof or the interpretation or construction thereof. 20.MODIFICATION OR AMENDMENT This Agreement and its attachments constitutes the entire Agreement of the parties on the subject matter hereof and may not be changed, modified, discharged or extended except by written amendment duly executed by the parties. Each party agrees that no representations or warranties shall be binding upon the other party unless expressed in writing herein or in a duly executed amendment hereof, or change order as herein provided. 21.APPLICABLE LAW This Agreement shall be deemed to have been made in, and shall be construed in accordance with the laws of the State of Illinois. Venue for the resolution of any disputes or the enforcement of any rights pursuant to this agreement shall be in the Circuit Court of Kane County, Illinois. 22.NEWS RELEASES The CONSULTANT may not issue any news releases without prior approval from the City Manager, nor will the CONSULTANT make public proposals developed under this Agreement without prior written approval from the City Manager, prior to said documentation becoming matters of public record. 23.COOPERATION WITH OTHER CONSULTANTS The CONSULTANT shall cooperate with any other consultants in the CITY's employ or any work associated with the PROJECT. 24.INTERFERENCE WITH PUBLIC CONTRACTING The CONSULTANT certifies hereby that it is not barred from bidding on this contract as a result of a violation of 720 ILCS 5/33E et seq. or any similar state or federal statute regarding bid rigging. 25.SEXUAL HARASSMENT As a condition of this contract, the CONSULTANT shall have written sexual harassment policies that include, at a minimum,the following information: A.the illegality of sexual harassment; B.the definition of sexual harassment under state law; C.a description of sexual harassment, utilizing examples; D.the CONSULTANT's internal complaint process including penalties; E. the legal recourse, investigative and complaint process available through the Illinois Department of Human Rights, and the Illinois Human Rights Commission; 5 F.directions on how to contact the department and commission; G.protection against retaliation as provided by Section 6-101 of the Human Rights Act. A copy of the policies must be provided to the Department of Human Rights upon request 775 ILCS 5/2-105. 26.WRITTEN COMMUNICATIONS All recommendations and other communications by the CONSULTANT to the PROJECT MANAGERS and to other participants which may affect cost or time of completion of the PROJECT shall be made or confirmed in writing. The PROJECT MANAGERS may also require other recommendations and communications by the CONSULTANT be made or confirmed in writing. 27.NOTICES All notices, reports and documents required under this Agreement shall be in writing and shall be mailed by First Class Mail, postage prepaid, addressed as follows: A.As to CITY: Sean Stegall City Manager City of Elgin 150 Dexter Court Elgin, Illinois 60120-5555 B.As to CONSULTANT: C4.‘-of /Ja7 S. /J''1,gy ,�c IZL/ SsAf. .o 3 ,tJe.sAri rk.. /L 6v/541 28. COMPLIANCE WITH LAWS, Notwithstanding any other provision of this CONTRACT it is expressly agreed and understood that in connection with the performance of this CONTRACT that the CONSULTANT shall comply with all applicable Federal, State, City and other requirements of law, including, but not limited to, any applicable requirements regarding prevailing wages, minimum wage, workplace safety and legal status of employees. Without limiting the foregoing, CONSULTANT hereby certifies, represents and warrants to the CITY that all CONSULTANT'S employees and/or agents who will be providing products and/or services with respect to this CONTRACT shall be legal residents of the United States. CONSULTANT shall also at its expense secure all permits and licenses, pay all charges and fees and give all notices necessary and incident to the due and lawful prosecution of the work, and/or the products and/or services to be provided for in this CONTRACT. The CITY shall have the right to audit any records in the possession or control of the CONTRACTOR to determine CONSULTANT'S compliance with the provisions of this section. In the event the CITY proceeds with such an audit the CONSULTANT shall make available to the CITY the CONSULTANTS relevant records at no cost to the CITY. CONSULTANT shall pay any and all costs associated with any such audit. Without limiting the foregoing, CONSULTANT hereby certifies, represents and warrants to the CITY that all CONTRACTORS employees and/or agents located in the United States, who will be providing products and/or services with respect to this CONTRACT,shall be legal residents of the United States. This agreement may be executed in counterparts, each of which shall be an original and all of which shall constitute one and the same agreement. For the purposes of executing this agreement, any signed copy of this agreement transmitted by fax machine or e-mail shall be treated in all manners and respects as an 6 original document. The signature of any party on a copy of this agreement transmitted by fax machine or e-mail shall be considered for these purposes as an original signature and shall have the same legal effect as an original signature. Any such faxed or e-mailed copy of this agreement shall be considered to have the same binding legal effect as an original document. At the request of either party any fax or e- mail copy of this agreement shall be re-executed by the parties in an original form. No party to this agreement shall raise the use of fax machine or e-mail as a defense to this agreement and shall forever waive such defense. The person signing THIS CONTRACT certifies that s/he has been authorized by the CONTRACTOR to commit the CONTRACTOR contractual and has been authorized to execute THIS CONTRACT on its behalf. Or FOR THE CITY: ��, , •- THE CONSULTANT: Sea r tegall / City Manager 7 { Project Approach industrial/Organizational Solutions, Inc. (IOS) is honored to have the opportunity to respond I to your Request for Proposals. This section of the proposal will detail our plan to provide a promotional written exam and assessment center for the rank of Sergeant in the Elgin, Illinois Police Department. Per the request of the City, IOS will detail a plan to deliver the following services: • Meet with Chief of Police, or his designee, to review the current job analysis and gain insight of the Police Sergeant duties in the Elgin Police Department. • Develop and administer a highly customized and job-relevant comprehensive written examination. • Develop and administer a highly customized and job-relevant comprehensive multi- exercise assessment center including highly job-specific behaviorally anchored rating criteria for the rank of Sergeant. • Recruit and train external law enforcement professionals to serve as assessors • Conduct an orientation session for the candidates to familiarize them with the assessment process. • Video record the entire assessment and rating process and provide all recordings to the City. • Provide final results within one week following the completion of the assessment center. • Provide candidate feedback, advising on both strengths and weaknesses, Kickoff Meeting We enter into a project with an expectation of implementing a specific best-practice methodology, however, we understand that every client has unique constraints and conditions that must be carefully considered in determining the proper assessment process design. Local laws, rules and procedures may cause us to alter our course. Also, there are often conventions that affect testing practices that cannot be easily changed or adapted such as Civil Service rules, collective bargaining agreements, etc. This is why we place a high value on conducting an initial kick-off meeting to learn about our client's particular situation and to consider the specific goals and expectations. This meeting is critical to ensure that we deliver a successful and beneficial service. The Project Director will first meet with your project manager and other designated personnel to discuss and determine specific objectives, personnel roles and project timelines. At this time the project manager will also learn about the duties and responsibilities of the role of Sergeant in the Elgin Police Department. Job Analysis Job analysis is the thorough investigation of the elements of a job. Generally, a job analysis is more extensive than a job description, which seeks to describe a job in summary/highlight form. 102 Copyright©2015,Industrial/Organizational Solutions,Inc CONFIDENTIAL—TRADE SECRET y The job analysis seeks to identify all job tasks that make up a job and the knowledge, skills abilities and personality characteristics (KSAPs) that are required to perform such job tasks. The job analysis is the foundation piece of a selection process because it serves to define the knowledge, skills and abilities that can be assessed by selection tools. In short, the job analysis is the basis by which we can demonstrate evidence of content validity. A job analysis does not need to be performed each time a selection process is developed. For this reason, it is useful to review the job analysis and determine its age and relevance. Generally, if the job analysis is less than five years old, it is reasonable to conduct a review of the essential tasks, knowledge areas, skills and ability to ensure they are still current and appropriate. If so, the existing job analysis can be used to develop selection tools and IOS will carefully document the job analysis review process to create a record of the review and updating process. To begin the job analysis, IOS will interview a stratified random sample of job incumbents for the target position. The sample will be stratified to ensure that a representative number of minorities and females are interviewed. Incumbents will be questioned about critical tasks, daily routines, essential job knowledge, critical skills and abilities, job challenges, critical job incidents, etc. In addition to interviewing incumbents, we will observe them doing their jobs. IOS will also interview a sample of supervisors to gain an additional perspective concerning job duties and responsibilities. Based on information obtained during the interviews/observations, IOS will develop a comprehensive job analysis questionnaire (JAQ). The JAQ contains a list of all potential tasks and KSAPs that are associated with the job. This questionnaire will be completed by a sample of incumbents to gain information about the nature of the job. Incumbents will rate each task or KSAP as to its importance (how important successful completion of the task is to the job) and frequency (how often the task is performed). Specific scales will be used to make these ratings. Prior to its administration, IOS will request that a group of subject matter experts (SMEs) who supervise the target position review the JAQ to ensure that it accounts for all job tasks and KSAPs, uses familiar language and is organized in a meaningful way. After obtaining feedback and making any necessary revisions, IOS will coordinate the administration of the JAQ. The JAQ will be administered to a large sample of incumbents. Incumbents will receive the JAQ along with a computer-scannable answer sheet on which ratings will be collected. We will ask the department to coordinate the completion of the JAQs and direct incumbents to turn the JAQ into a supervisor within approximately two weeks. At the end of this period, all JAQs will be returned to IOS for analysis. JAQ data gathered from incumbents will be analyzed to identify those tasks and KSAPs that are essential to the target job. This will be done by combining the "importance" and "frequency" ratings. Generally, the ratings given to any task or KSAP will be weighted using the following formula: .666(importance rating) + .333(frequency rating). This formula will yield an "essentiality" rating. All those tasks and KSAPs with a high enough essentiality rating will be considered essential job tasks and will be used to define the target job. 103 Copyright C)20!S.Industrioll0rganizar onoi Solutions,Inc. CONFIDENTIAL—TRADE SECRET Using the list of essential job tasks and KSAPs, IOS will engage a process to link essential tasks to the KSAPs that are required to accomplish these tasks. This linkage process will ensure that the job tasks are related to specific, measurable knowledge areas and KSAPs that can be assessed using conventional tests and assessments. Essential KSAPs will later be used as the basis for developing written exams, structured assessments, assessment centers or other types of selection tools. The KSAPs will define the measurable dimensions of each selection tool. The job analysis process will yield a comprehensive list of essential job tasks and KSAPs. A report will be completed to outline the methodology used during the job analysis and to summarize the outcomes and ratings of the job analysis process. This report will document the following methodological elements: • Job observation/interview sampling and outcomes • Job analysis questionnaire development and review • Job analysis questionnaire sampling, distribution and analysis • Task-KSAP linkage • Essential task and KSAP results Promotional Job Knowledge Examination Development and Validation The promotional job knowledge examination is a written tool that is specifically designed to assess essential, job-related knowledge. To this end, exam questions will be drawn from relevant, job-related sources of knowledge (i.e., text books, policy, rules, law, etc.). The job knowledge exam will be supported by content validity evidence. This evidence will be established through a thorough linkage of essential knowledge areas identified in the job analysis to the measureable dimensions of the exam, and through subject matter expert involvement in the development of the exam plan and review of exam items. Exam Plan and Reference List Development The first step in the development of a written job knowledge exam is the identification of the knowledge areas that should be assessed by the exam and the source, or reference materials from which this knowledge can be drawn. An IOS consultant will review the target job description/job analysis to identify those knowledge areas that should be assessed by the written exam. IOS will provide a recommendation as to the knowledge areas that should be assessed. At this point, IOS can work with department subject matter experts to identify appropriate sources to assess the suggested knowledge areas, or the department can independently recommend sources that assess the suggested knowledge areas. The following tables identify common knowledge areas that are assessed by public safety written examinations: 104 Copyright©2015,IndustriollOrgonizationol Solutions,Inc CONFIDENTIAL—TRADE SECRET General Category Law Enforcement Exams Technical Field Operations Knowledge Criminal Investigation CompSTAT Community Policing ' Ethics in Policing Multicultural Law Enforcement Management Supervision Knowledge Administration Leadership/Motivation Internal Sources Policy, Procedure, or Operating Guidelines Collective bargaining i agreement State Law Once sources are selected from which to draft examination questions, it is necessary to create an exam plan that identifies the percentage of items that will be drawn from each source/knowledge area, the specific content that will be focused on, and the information that will be provided to candidates in the reference/study list. A subject matter expert (SME) meeting will be arranged to review these recommendations and finalize the selection of knowledge areas and sources/references. During this Exam Pian Development Meeting, SMEs will be asked to assist in identifying the relative importance of the knowledge areas that will be assessed by the exam. These ratings will be used to establish the number of items that will be drafted to assess specific knowledge areas. In addition, SMEs will review reference/source materials and assist in identifying the chapters and sections that are most relevant for the target position. This step will ensure the test is as job-related as possible and will allow candidates to focus specifically on essential information contained in study materials. In short, this meeting will allow department subject matter experts to ensure that the exam plan and the candidate study/reference list meet their expectations. Following the Exam Plan Development Meeting, IOS will generate the final exam study list that will contain a list of sources that will be used to create the exam and the specific chapters/sections that candidates should review. This list will be submitted to the department for distribution to eligible candidates. IOS will also produce a candidate preparation document that provides helpful strategies for study and preparing for the written examination. This document can be distributed to candidates or posted on the agency website. *Note: The department may wish to develop the reference list on their own. In this case, IOS will be charged with ensuring that essential knowledge areas are being assessed via the chosen reference list. IOS may also still need to work with SMEs to identify the specific chapters and sections of source material from which to draw exam items. 105 Copyright C 2015.IndustrioflOrgonizauona!Solutions,Inc CONFIDENTIAL—TRADE SECRET Question Development and Review IOS technical writers have advanced degrees in writing and are extensively trained, mentored and led by Ms. Karen Steiger, M.S., our Senior Technical Writer/Manager of Technical Writing, who has been with IOS for over a decade and has devoted her professional career to writing public safety examinations that measure relevant job-related content areas. Ms. Steiger has personally created over 200 promotional examinations for public safety agencies and has overseen the development of countless others developed by her staff. Further, our technical writers are responsible for responding to candidate challenges and upholding the reliability of the exams. For anyone who has watched candidates sit through a 2 1/2 hour long, 100 item, written exam and worked through a challenge process and air their concerns regarding each specific item, it becomes abundantly clear that the critical role of written exam item development is the backbone of a successful and fair promotional process. Mr. Legel's insistence on maintaining a highly educated and experienced team of technical exam writers is why IOS has an unblemished litigation history. Using the detailed examination plan that was developed, IOS team of highly-trained technical writers will draft exam questions from the identified source materials. Our technical writers take particular care in identifying the most important topics and concepts from source material and drafting questions that are highly job-related. In addition, the technical writers will draft questions of varying degrees of difficulty to ensure that the test is able to discriminate between candidates' levels of job knowledge. Approximately 125 questions will be drafted based on the examination plan. These questions will later be reviewed and distilled down to the best 100 questions. Each question will be drafted in multiple-choice format. Once test questions are developed, an IOS consultant will review each item to ensure that it is job-related, in-line with the exam plan and is free from any apparent bias. The development of good test items is a critical element to the success of any job-knowledge exam. For this reason, we take care to make sure the following criteria are met: • Focus on essential information: Our item writers rely on the job analysis information gathered by our consultants and base the content of an exam on the test blueprint (or exam plan) that has been developed with SME input and I/O psychology best practices. The item writing team combs selected sources to identify truly job-related, important information from which to draft questions. We try to focus on "action," when possible (i.e., what should an employee do in a given situation, based on policies and procedures, state and federal law, and current management theory). • Multiple-choice test items: "Scan-form"-style testing is efficient and easy to score, but it has a potential weakness — if candidates can identify the correct response through deductive reasoning, then they are demonstrating deductive reasoning ability rather than demonstrating the desired knowledge. This is why it is critical that answer options be well constructed and that there are enough answer options such that the candidate does not a stand a reasonable chance of guessing the right answer. o True/false formats are simply insufficient. o All tests should have at least four answer options. We create variety by developing both fill-in-the-blank items and "situational" items (or vignettes), 106 Copyright®20 S,IndusieoI/Orgomzotanol Solutions,Inc CONFIDENTIAL—TRADE SECRET asking candidates to put themselves in the position of the employees described in the question. • Appropriate difficulty: We ensure that questions developed for a job-knowledge exam are of the appropriate difficulty for the rank or position being tested. Our item writers carefully search through source material for the information most directly related to the job in question. o We avoid "tricky" items that would confuse those who are minimally qualified for the position or rank in question. o After the items have been developed, we rely on SME input when possible to determine that the content of the items is appropriate for the group of candidates being tested. o Another way that we ensure appropriate difficulty is to ensure that answer options are similar to each other in style, structure and length. • One correct response: We carefully craft questions such that there is one correct response and that the remaining responses are incorrect. We consider accuracy to be absolutely crucial in our job-knowledge exams and thoroughly review items both internally(and with the client, when possible). • Avoidance of negatively worded stems/response options: Negatively worded questions and answer options are confusing. In order to ensure that we are measuring knowledge in a straightforward manner, we try to avoid the use of negatively worded items. • Avoidance of overly complex stems/response options: Again, we want questions to be simple and easy to understand. For this reason, we avoid confusing or overly lengthy questions while ensuring that the item remains accurate. Our item writers are trained to create clear and focused question stems and distracters. • Avoidance of "all of the above" or "both c and d"-style answer options: This style of questioning is generally overused and tends to reduce the psychometric quality of a test. • Cultural and gender sensitivity: Our item writers are trained to use appropriate terminology and language when referring to members of any protected group, referring to their issues and beliefs with respect. When sensitive or controversial subjects must be discussed in our exams, these topics are described in an objective manner. We take care to ensure that personnel are described in gender-neutral, ethnically diverse terms IOS will work with the department to assemble an SME panel to review test questions. The purpose of this review is to ensure that test questions are of high quality and to collect SME rating to support the validity of the test and to enable the determination of an appropriate test cut-off score. SMEs will be asked to review each item and offer any pertinent feedback/suggestions. In addition, SMEs will provide a Content Validity Ratio (CVR) Rating and an Angoff Rating. The CVR rating will provide evidence of the degree to which the content of 107 Copyright t 2013,Industrial/Organizouonal Solutions.Inc CONFIDENTIAL—TRADE SECRET the test mirrors critical content of the job. The Angoff Rating will identify the difficulty of each test item, thus indicating the overall difficulty of the exam. The Angoff process allows for the selection of a cut-off score that equates to minimally qualified levels of job knowledge. While this review is optional, the data collected during this process is highly valuable to the defensibility of the written exam. We recognize that agencies are often required to use a pre-determined cut-off score. In these cases, the Angoff process is still appropriate but must be used in a different manner. As a matter of legal defensibility, it is necessary to demonstrate that a cut-off score, even if mandated, equates to minimally qualified job knowledge levels. In a case where an agency is required to use a 70 percent cut-off score, for example, IOS will work toward selecting test items whose composite Angoff score is approximately 70 percent. In this manner, the cut-off score will be in-line with the test's difficulty level. Data from the item review meeting will be used to select the 100 test questions that will be included in the written exam. These questions will be professionally compiled into a test booklet. Questions from the same source material will be grouped together under a title heading such that candidates are aware of where the item was drafted from as they take the test. IOS will produce all necessary test booklets and scannable answer sheets. Exam Scoring, Results Generation and Analysis IOS will create an exam specification for the written exam that identifies the correct answer for each item, the source citation from which the question and correct answer are drawn and the knowledge area that the question assesses. This document will be used to generate a computer-based scoring program that will be used to score the exam. Each completed exam answer sheet will be scanned and processed by an optically scoring machine. Data from this machine will be manually reviewed for accuracy and scored using a computer-generated scoring program. The project consultant will conduct an item analysis process to ensure that each item of the test functions in a reasonable manner. Specifically, IOS will investigate the following: • Adverse impact ratio • Individual item data: item-total correlations, mean (p-value), standard deviation, frequency analysis to check for item with multiple correct responses, etc. If any items appear to be problematic (a large proportion of candidates get the item incorrect or there appear to be multiple correct responses based on candidate data), IOS will engage a discussion with the department and make a recommendation for resolving any issues. Final test scores will then be calculated. Exam results will be presented in a database format such as Microsoft Excel®and submitted to the department per the department's specifications. Test Challenge/Appeal Process It is commonplace to conclude an examination by allowing candidates to participate in a challenge/appeal process where candidates can scrutinize test items and draft appeals to the 108 Copyright©201 S,IndustriallOrganizoconal Solutions.Inc CONFIDENTIAL—TRADE SECRET validity or accuracy of items. IOS generally handles appeals processes in one of two ways. The first option is preferable, but we work with the department to determine the best solution based on standard procedures, convention, circumstances, etc. Following are the two options for challenge processes; I. Immediately following the administration of the written exam, candidates are provided with an answer key, their test booklet and a copy of their answer sheet. Candidates are allowed to manually score their exam while being proctored by an IOS representative. Candidates are instructed that this score is preliminary and that tests will be carefully scored at a later time. Following this scoring period, candidates are instructed that they will be afforded the opportunity to file appeals to test items if they so choose. As candidates have been able to review the questions (in the test booklet) and make a determination of which items they answered correctly, they can reasonably determine whether or not they wish to file any appeals at that time. Candidates will be allow a fixed period of time (usually one hour) to draft written appeal on appeal sheets provided by IOS. Candidates will be allowed access to the source materials as well as an exam specification that indicated a citation for each test item. IOS will collect all appeals and prepare a written response to these appeals. Appeal responses will be turned over to the agency with a recommended action for each item. IOS will engage a discussion with the agency to determine if any changes should be made to the scoring of test items. Once a determination is made, final scores will be calculated and submitted to the agency. 2. Within a few days of the test, candidates will be allotted a structured time period to review the test booklet, copy of their answer sheet, exam specification, answer key and test source materials. Candidates will be allowed to draft appeals which will be submitted, reviewed and responded to in the manner described above. The agency will likely be responsible for monitoring candidates during these structured time periods. Assessment Center Development and Validation An IOS assessment center is a battery of job simulations/exercises geared toward measuring essential management and supervisory skills of promotional candidates. Common exercises include one-on-one role-plays, group presentations, tactical simulations, in-basket/in-box simulations, and written problem solving exercises. An IOS assessment center exercise puts the promotional candidate in a situation that is similar to a function he/she will perform on the job. Assessors, who all have industry-specific management experience, rate the candidate on essential management and supervisory skills and abilities that will determine the candidate's success on the job. The assessors use specific rating criteria that were developed by experts in the industry. The result of the 105 assessment center is a list of candidates in rank order according to who is most likely to perform at the highest level in the target position. The assessment process also yields an abundance of feedback from the assessors about candidate performance that can be used for developmental purposes. A comprehensive IOS assessment center will incorporate the following components: 109 Copyright 0 2015,Induur,all0rganizotonal Solutions,Inc CONFIDENTIAL—TRADE SECRET • Assessors who are experts in their industry and are specifically trained to assess candidate performance • identification of specific skill and ability dimensions with associated behavioral criteria • A standardized administration schedule • Relevant, simulation-style exercises IOS' assessment centers are simply the best way to predict how candidates will react to/handle specific job situations. The IOS assessment center method also allows for a broad range of skill assessments. We are able to use a variety of simulations that assess many different skills and abilities. This is a desirable feature as your goal in promotions is to select the most well- rounded candidate, Candidates are also more prone to feel that IOS assessment centers are fair evaluations of their promotional suitability as it is readily apparent that the assessment center is a valid and comprehensive way of making a promotional decision. Finally, an IOS assessment center is also a developmental tool. The results of an assessment center can be tailored to assist an individual candidate toward improving performance in a specific area. Data from a large number of candidates can also be used to help agencies identify training needs. The assessment center will be supported by content validity evidence. This evidence will be established through the clear linkage of the job description/job analysis to the skills and abilities that will be assessed in the assessment center and by the linkage of essential job tasks to the simulation exercises that are selected. Furthermore, SMEs will assist in reviewing each exercise and rating criteria to ensure local validity. Exercise Selection and Development IOS will first work to identify the skills and abilities that should be assessed by the assessment center by reviewing a current job description/job analysis for the target position. Based on this review, IOS will provide a recommendation concerning the skills and abilities that should be assessed and the types of exercises that should be used to assess these skills and abilities. In order to refine this recommendation, a consultant will meet with department subject matter experts (SMEs) who hold the target rank and/or supervisor the target rank to identify job duties and critical scenarios that would constitute valuable assessment exercises. IOS will use these ideas and suggestions to craft exercises that are locally specific. SMEs will also review skills and abilities that will be assessed to ensure that the assessment is comprehensive and addresses any concerns of the department. While the specific content of the assessment center will be determined during the course of SME meeting, the following tables provide information about the types of exercises that can be created and common skills and abilities that are assessed by our assessment centers. One-on-One Role-Play Scenarios Exercise Subordinate counseling(problem employee) Description The candidate plays the role of a supervisor who must meet with a subordinate employee(role- player) that has one of the following types of problems: a performance deficiency, a recent conflict with a citizen, personal problems or difficulty getting along with coworkers. The candidate will be given introductory information about the subordinate and the problem and will be allowed a brief period of time to prepare for the meeting. The candidate will then meet with the subordinate and attempt to deal with the issue at hand to the best of his/her ability. The I10 Copyright©201 S,industrial/Organizational Solutions,Inc CONFIDENTIAL—TRADE SECRET candidate will likely meet resistance from the subordinate that will require him/her to exercise supervisory skills. The successful candidate will mitigate the problem and devise an appropriate solution. This exercise can be coupled with the citizen meeting exercise. This paring would require the candidate to gather specific information about a citizen conflict from the citizen and then use this information to deal effectively with the subordinate. Themes This exercise can deal with counseling subordinates on any of the following issues: performance deficiencies, a run-in with a citizen, employee dislike his/her job, depression, marital problems, drug/alcohol issues,or inability to get along with coworkers. Time There is a 10 to 15 minute preparation period prior to the 15 to 20 minute exercise. Resources This exercise requires one skilled role-player/actor to play the role of the subordinate and three qualified assessors.This exercise will occur in a small meeting room. Dimensions ' Interpersonal skills, problem analysis/problem solving, judgment and reasoning, decision-making ability, management and supervisory skills,composure,conflict resolution, oral communication. Exercise Citizen meeting Description The candidate plays the role of a supervisory o icer that is meeting with a citizen.The meeting allows the citizen (role-player) to address a concern that he/she has about the department in general or more specifically about one of the candidate's subordinates. The goal of the meeting is to work effectively with the citizen to solve his/her problem or collect information toward bettering the department.The candidate will be given a preparation period to review the task at hand and to devise a game-plan. The candidate will then meet with the citizen. The citizen will follow a specific script that will cause the candidate to have to adapt his/her approach and CO think on his/her feet. The successful candidate will gather necessary information, instill confidence in the citizen and sec up a system to gauge progress and follow-up with the citizen. This exercise can be used as a precursor to the subordinate counseling role-play. This pairing would require the candidate to use this meeting to gather facts in order to guide the direction of the meeting with the subordinate. Themes The topic of the meeting can be: inappropriate treatment of the citizen by a department member(typically the candidate's subordinate), the citizen's specific concerns about recognized deficiencies in the operations of the department, or the collection of feedback from the citizen about proactive improvements that could be made to increase levels of customer service. Time There is a 10 CO 15 minute preparation period prior to a 15 to 20 minute interaction. Resources This exercise requires one skilled role-player/actor to play the role of the subordinate and three qualified assessors. This exercise will occur in a small meeting room. Dimensions Interpersonal skills, problem analysis/problem solving, judgment and reasoning, decision-making ability, conflict resolution,public relations skill, oral communication. Exercise Subordinate performance review Description The candidate plays the role of a supervisor that is tasked with presenting a performance appraisal to a subordinate. Prior to the exercise the candidate will be provided with the subordinate's personnel file. The subordinate will typically have a number of performance deficiencies that have not improved over time.The role-player, playing the subordinate role,will be evasive and, at times, combative about apparent performance weaknesses. The candidate's objective will be to analyze the subordinate's personnel file in order to successfully identify and work through the subordinate's deficiencies. The candidate will have to demonstrate a large degree of assertiveness as he/she counsels the subordinate regarding deficiencies and identifies plans for improvement. The ideal candidate will exercise management skills while maintaining composure in a difficult situation. Themes The subordinate typically has a few areas of performance deficiency that must be addressed in the course of the performance appraisal. These may include: tardiness,improper completion of paperwork.poor attitude,and complaints from coworkers. Time There will be a 15 to 20 minute preparation period for the candidate to review the subordinate's personnel file and devise a structure for the performance appraisal meeting. The meeting will then last approximately 15 to 20 minutes. Resources This exercise requires one skilled role-player/actor to play the role of the subordinate and III Copyright©2015,Industrial/Organizational Solutions,Inc. CONFIDENTIAL—TRADE SECRET three qualified assessors.This exercise will occur in a small meeting room. Dimensions Management and supervisory skills, judgment and reasoning, decision-making ability, planning and organization,interpersonal skills, conflict resolution, oral communication. Presentation-Style Role-Plays Exercise Training presentation Description The candidate will be tasked with preparing a training presentation to be delivered to a group of subordinates. If the candidate is a police promotional candidate, the training session will typically be a roll-call training brief. In the case of a fire promotional candidate.the training will be similar to in-service training. In either case, the candidate will be provided information about a topic that the subordinates must be trained on. The topic will be job related and will be something that the candidate can become familiar with in a short period of time. After a brief time to organize a training session and possibly create visual aids, the candidate will present the lesson to a group of three subordinates. The subordinates will either be role-players or the assessors. During the course of the training the candidate may interact with the role-players, or he/she may require the role-players to participate. The role-players will also be free to ask questions and respond as would be conventional in this type of setting. The successful candidate will deliver a well organized and thoughtful training presentation in a manner that is practical and engages the audience. Supervision skills and interpersonal ability will be critical factors as the candidate may have to deal with unruly subordinates. Themes Police supervisory themes include: use of force training, pursuit training, community initiatives training and policy training. Department-specific topics can also be used to further customize the process. Time There will be a 15 to 20 minute preparation period for the candidate to review the training material and outline the training agenda. The training session will then last approximately 15 to 20 minutes. Resources This exercise requires three qualified assessors and possibly one or two skilled role-players. The role-players will play the role of subordinates and will have a more interactive role than that of the assessors'.The use of role-plays will make the exercise more realistic and increases the quality of the assessor's ratings as they are not burdened with interacting during the exercise. Use of assessors as role-players is, however, a less expensive alternative. A dry erase board or flip chart will be provided to the candidate to enable him/her to better structure the training. This exercise will occur in a medium sized meeting room with chairs arranged in a classroom style setting. Dimensions Planning and organization, oral communication, interpersonal skills, management and supervisory skills. Exercise Community presentation Description The candidate is instructed that he/she must deliver a presentation about the department to a group of citizen who are interested in learning more about the department. The group of citizens will be played by either the assessors alone, or the assessors accompanied by additional role-players. The presentation is usually made before a group of citizens or at a city counsel- type meeting. The candidate is typically informed that the chief was scheduled to make this presentation, but he/she was called out of town and has asked the candidate to fill in for him/her. The candidate will be given a brief period of time to organize his/her thoughts and outline the presentation. There are no materials given to the candidate to assist in the development of the presentation.The candidate must develop and deliver a presentation on the specified topic based solely on his/her understanding of the department. The candidate will be able to use visual aids that he/she creates using flipcharts or dry-erase boards. The successful candidate will develop a well-organized and meaningful presentation that provides the audience with valuable information about how the department is operating within their community. Following the presentation the audience will ask questions related to the candidate's topic. His/her ability to answer questions in a professional and responsive manner will be part of the assessment. 112 Copyright©2015.industrial/Orgonizotionol Solutions,Inc. CONFIDENTIAL-TRADE SECRET Themes The presentation can be related to: department operations, response capabilities, the services provided by the department, the goals/mission of the department. or a recent improvement made within the department. Time There will be a 15 to 20 minute preparation period for the candidate to review and to prepare an outline for his/her presentation.The meeting will then last approximately 15 to 20 minutes. Resources This exercise requires three qualified assessors. The assessors will serve as assessors but will play the role of citizens.They will ask questions of the candidate at the end of the presentation. Additional role-players may be used in this exercise, but it is not necessary. A dry erase board or flip chart will be provided to the candidate to enable him/her to better structure the training. This exercise will occur in a medium sized meeting room with chairs arranged in a classroom style setting. Dimensions Planning and organization, oral communication, interpersonal skills, management and supervisory skills, leadership. Written Exercises Exercise Written problem-solving exercise I Description This exercise requires the candidates to compose a written narrative related to an important departmental issue, The goal of this exercise is to outline the thought process that is used in solving a problem or reasoning through a difficult issue. This exercise is useful for evaluating basic problem solving skills, judgment and reasoning, and written communication. This exercise can also be administered to many candidates at the same time and does not require the use of role-players. Themes Following are optional topics that can be used: • The candidate must identify critical issues facing the department and how he/she would handle these issues. • The candidate is faced with a difficult personnel issue (subordinate performance deficiency or sexual harassment) and must outline in a narrative how he/she would handle this issue. • The candidate will be asked to make suggestions for improving and better implementing a current departmental policy. The policy will be selected by the agency. Time The candidate will be allowed one to two hours, depending on the complexity of the questions, to complete this exercise. Resources A large classroom will be necessary to administer this exercise to a large number of candidates. Dimensions Judgment and reasoning, written communication, planning and organization, management and supervisory skills, Exercise Incident report proof-reading exercise Description This exercise uses copies of your agency's incident report to assess the candidate's ability to proof read an incident report for serious errors. A sample report will be created containing errors in documenting critical information and general grammatical and writing errors. The candidate will be asked to proofread the document, identify errors and suggest changes for improving the report. Themes Any standardized report can be used for this exercise. Time The candidate will be allowed 15 to 20 minutes to complete this assignment. Resources A large classroom will be necessary to administer this exercise to a large number of candidates. Dimensions Problem analysis/problem-solving,written communication. In-Basket Exercise Exercise In-basketlin-box Description The candidate is told that he/she was recently promoted and has inherited the former incumbent's in-basket. It is imperative that he/she go through the in-basket and prioritize and respond to as many items as possible. There are approximately 10 to 20 memos, letters, email 113 Copyright©2015,industnollOrganizauonol Solutions,Inc CONFIDENTIAL—TRADE SECRET messages, phone messages, and other correspondence of varying importance and urgency that deal with personnel issues, scheduling, citizen concerns. planning and development, departmental issues, etc. The successful candidate will properly prioritize the items and provide appropriate written responses for how the issue at hand would be dealt with. In some cases the candidate may have to write a letter and in others simply detail an action-plan that he/she would follow to deal with the situation. There may be an additional follow-up period when assessors question the candidate regarding his/her chosen course of action. Themes There are a variety of in-basket items that are part of this exercise: however,depending on the role of supervisors in your department, more items of the following variety can be incorporated into the exercise in greater quantity: employee misconduct issues, citizen complaints, service- oriented requests.employee counseling issues, or scheduling/administrative issues. Time There is a 10 to 15 minute instruction period prior to the start of the exercise and, depending on the number of in-basket items, this exercise will last one to two hours. Resources I The in-basket exercise is rated by two or three assessors following the administration period. This exercise requires a classroom-style room for administration to multiple candidates. Dimensions Planning and organizing, problem analysis/problem solving, judgment and reasoning, decision- making ability,management and supervisory skills,written communication. Situational Structured Oral Assessment Exercise Situational oral assessment Description The situational oral assessment is a panel-style interview that requires the candidate to consider how he/she would respond to a variety of situations that he/she might encounter on the job. During the interview assessors will pose a variety of job-related questions to the candidate and will rate the candidate's response using behavioral rating criteria. The questions can cover a variety of job-related scenarios or interpersonal situations.This exercise is not intended to be a role-play, but certain questions can be converted into role-play scenarios to maintain the I "simulation"feel of assessment centers. Themes The questions contained in the oral assessment can cover three to four of the following topics: dealing with a problem employee, controlling a tactical situation, managing operations of the agency,or improving ones own skills. Time This exercise typically lasts for 20 to 30 minutes. Resources i A small meeting room will be required for this exercise. Dimensions I Depending on the questions used,any performance dimension can be assessed in this format. Performance Dimensions The assessors that are used to rate candidate performance are guided by specific performance dimensions with behaviorally anchored criteria. These criteria allow the assessor to look for certain behaviors, actions and thought processes. This method of rating performance ensures consistent and fair ratings. IOS has constructed a comprehensive performance model through years of experience developing and conducting assessment centers for public safety agencies of all sizes. Following is a comprehensive list of the performance dimensions that can be assessed. The behaviorally anchored criteria that accompany these dimensions are specific to the exercises that are used in the assessment center. This information is provided to demonstrate the model that IOS commonly uses. We expect that the performance model will be modified based on the job description for the target rank. _General Management Dimensions Composure:The ability to work under stress and perform in unpleasant or traumatic circumstances.The ability— to remain calm in stressful situations. The ability to handle stressful situations appropriately, remaining poised and professional.The ability to maintain emotional control in stressful situations. 114 Copyright©2(115,Industrial/Organizational Solutions,Inc. CONFIDENTIAL—TRADE SECRET Conflict Resolution: The ability to de-escalate tense situations and quiet potential disturbances. The ability to mediate interpersonal or physical conflict between individuals. Decision-Making Ability:The ability to understand when a decision must be made and the willingness to make a decision. The ability to quickly and efficiently determine an appropriate course of action to target a particular situation. ' Honesty and Integrity: The ability to act in an honest and fair manner. Willingness to accept responsibility for I actions when things go wrong. The ability to display a high degree of integrity and professionalism in action and i word. I Interpersonal Skill: The ability to listen to others and be considerate of the concerns of others. Ability to successfully and appropriately handle interpersonal interactions with others to gain trust, respect and mutual understanding. The ability to act with tact and diplomacy in dealings with the public - keeping public-relations concerns in mind when interacting with citizens, other agencies and the general public. The ability to counsel, support and be empathetic toward others. Ability to maintain positive work relations with others and interact with people of diverse backgrounds. The ability to use appropriate nonverbal cues to better convey intention of message. Judgment and Reasoning:The ability to accurately perceive the important elements of a situation,evaluate the situation and determine plausible courses of action that would bring about a desirable result. The ability to reason through a particular problem and decipher a logical course of action. The ability to use common sense and intelligence in handling day-to-day activities,problems and decisions, Leadership: The ability to set a proper tone for the functioning of the organization. Ability to serve as a role model in behavior, practice and word. The ability to motivate the organization as a whole to achieve greater ! efficiency and a higher level of purpose. Management and Supervisory Skills:The ability to direct and guide personnel in the accomplishment of goals and tasks. (Includes skill in monitoring activities and evaluating results. Expectations and standards are clearly communicated, and consequences exist for insufficient performance.) The ability to assert proper authority and I be assertive while maintaining the trust and respect of subordinates. The ability to motivate and counsel employees to greater performance. Skill in prioritizing and delegating the work of others. Oral Communication: The ability to speak in a clear, concise. understandable and appropriate manner. The i ability to deliver a message in such a way that others clearly understand its meaning. Appropriate use of nonverbal cues such as good eye contact,posture,etc. Planning and Organization: Skill in establishing a course of action for oneself and others to accomplish goals. Skill in properly planning assignments, work duties and tasks; appropriate allocation of resources to ensure efficient, timely and smooth department operations. The ability to anticipate future needs and plan for the i allocation of resources to meet those needs. Problem AnalysislProblem Solving: The ability to quickly perceive problem areas, identify options and the potential impact of various solutions. The ability to determine courses of action or solutions to problems. The ability to implement targeted solutions. Public Relations Skill:The ability to deal with the public in such a way as to protect the image and reputation of the department while providing the public or individuals with necessary or requested information. The ability to deal prudently with information such that the integrity of individuals or the department as a whole is not compromised. Self-Motivation: The ability to remain motivated to perform one's job despite difficult circumstances. distractions or monotonous work, The ability to take charge when necessary and conduct tasks without being told to do so.The ability to work without supervision and accomplish tasks without prodding. Teamwork Orientation: The ability to work with others in a positive,goal-oriented manner. Willingness and ability to accept a particular role in a team and selflessly carry out that role. Willingness to actively support command staff policies and decisions. Written Communication: The ability to convey written messages in a clear, concise and easy-to-read format. The ability to clearly express ideas in writing to convey intended meaning. The ability to use correct spelling, grammar, syntax and proper subject-verb agreement. Exercise Review IOS will develop the selected assessment exercises in their entirety, including candidate instructions, role-player scripts (if applicable), assessor rating guidelines, scoring scales, etc. 115 Copyright @ 2013,industrial/Organizational Solutions,Inc CONFIDENTIAL—TRADE SECRET Once exercise development is complete, IOS will work with the department to convene an SME meeting to review each exercise and its rating criteria. The purpose of this review will be to ensure that the details of each exercise are specific to the organization and operations of the department and to ensure that rating criteria are consistent with performance expectations of the department. During this review SMEs will also provide input to help define minimally acceptable performance on each exercise. This input will assist in the calibration of rating scales and will be critical during the training of assessors. Assessment Center Scale and Scoring All assessment center scores will be compiled in scoring booklets used by assessors during the course of the assessment process. These booklets will contain all behavioral rating criteria for each exercise as well as the scoring scale that will be used by assessors. The scale normally used by IOS is a 10-point scale that defines '6' as minimally qualified. The 10-point scale is used because it easily translates into a percentage score. The minimally qualified designation creates an effective cut-off score. During assessor training, assessors are informed that any score below '6' is considered less-than-qualified. Therefore, the assessment process has a built-in cut-off score that assessors are keenly aware of and use to distinguish qualified candidates from unqualified. 2 3 4 1 5 6 7 8 9 I0 —Exceptionally Very Poor Poor Below Adequate Above Strong Very Exceptional Poor Average Average Strong IOS' personnel enter all score data collected from the assessment center process into a database. This process is performed once and then independently verified to ensure accuracy. Then, scoring equations are applied to the data set and subsequently, independently evaluated for accuracy. Obviously, it is of paramount importance that scores be correctly calculated and assigned. Given the vast number of data points collected in an assessment center, we take great care in compiling and calculating scores. Final scores will be provided to the City within one week following the completion of the assessment center. Candidate Orientation Preparedness plays a large role in creating fair promotional processes. Because many candidates that will compete in an assessment center have previously experienced assessment centers, it is helpful to provide an orientation process that allows first-time participants to gain the insight that seasoned candidates have obtained through experience. Further, assessment centers from year to year are often conducted by different consulting firms that often conduct assessment centers differently. It is important to clearly communicate to candidates how the assessment center will be conducted and how they will be evaluated. By supplying candidates with an opportunity to learn about and prepare for the assessment center, we will increase the reliability of candidate scores and reduce the risk of group score differences in the assessment center process. IOS advocates the uses of a candidate orientation to create transparency and to provide helpful information to candidates. Specifically, we recommend that a live orientation presentation be 116 Copyright©2015,Industrial/Organizational Solutions,Inc. CONFIDENTIAL—TRADE SECRET delivered. This presentation can be video-recorded for presentation to candidates who were not able to attend the original orientation. Should additional sessions be necessary, these can be conducted based on the need of the individual agency. The following are common components of our candidate orientation guide: • A general introduction to assessment centers • A review of the assessment center development process • Review of the types of exercises that can be included in the assessment center • An in-depth discussion regarding the nuances of individual exercise types • Review of the performance dimensions assessed • Review of the scoring system used for the assessment center • A discussion concerning assessors and assessor training • Advice for preparing for assessment centers • A brief discussion of common mistakes and misconceptions related to assessment centers (these will not cover specific performance expectations) • A review of the rules governing the assessment center • Question and answer session Assessor/Role-player Recruitment Assessors are recruited based on a recruitment plan that is accepted by the department/agency. This plan include the target geographic (national vs. state), the acceptable rank/level of assessors, gender and race diversity goals, personal qualifications, and any other criteria valued by the department. Following are the common rules that IOS imposes on assessor recruitment: • All assessors must be one rank/level above the rank for which the assessment center is being administered. • All assessors must possess at least two years tenure at the minimally required rank. • All assessors should be from a similarly sized agency that is organized in a substantially similar manner. • Racially diverse assessors will be sought to participate in all assessments. Our standard goal will be that one third of all assessors be classified as racial minorities. • We will seek maximum gender diversity in assessor recruitment. • All assessors should have no connection to the police department or any of the candidates. Role-players will also often be required during the assessment center process. IOS prefers to use professional actors to perform this function. Many consultants tend to use law enforcement of fire service personnel to serve as role-players. While we understand the convenience associated with this practice, we disagree with the appropriateness. Professional actors are far superior for a number of reasons: • Consistency/standardization: A professional actor is able to play the same role over and over again without altering presentation. This is a difficult task that requires training and 117 Copyright©2015,lndustrlallOrgonrzotional Solutions,Inc CONFIDENTIAL—TRADE SECRET endurance, but it is absolutely essential to the standardization and reliability of the process that each candidate experience the same scenario. • Role-fidelity: Assessment roles are scripted to accomplish a very specific goal. Professional actors are able to aptly model the role that was intended. Often, amateur actors will morph the role to a persona that better suits their personality. This practice can be detrimental to the assessment process as the primary performance dimensions may not be assessed as reliably in this instance. • Realism: Simply put, professional actors do a much better job of making the scenario 'real' for the candidate. Given the challenge of role-playing, most candidates find it difficult to treat a scenario as 'real.' A gifted actor will help draw the candidate into the process by making the exchange high-fidelity. • Reliability: Ultimately, the superior performance of professional actors, increase the accuracy of candidate performance/scores and give us more confidence that we have fairly and accurately assessed the true ability level of the candidate. IOS will recruit role-players from professional acting guilds and talent organizations. We have established partnerships with talent agencies in various part of the country from which we recruit actors that have been used in past assessment centers. Where it is cost prohibitive to bring actors in from distant locations, we will contact local acting guilds to recruit role-players. Assessor Training IOS will conduct in-depth assessor- and role-player-training workshops to prepare each assessor and role-player to participate in and score the assessment-center exercises. We will require approximately six to eight hours to train the assessors and role-players. This training is generally conducted the day prior to the administration of the assessment center to candidates. IOS will provide multiple consultants to facilitate training multiple panels at the same time. Following is an outline of the training that is provided: Role-Players: During this time role-players will be briefed on their characters and the requirements of the roles. Role-players will be allowed ample time to ask questions concerning their roles and to practice their roles with IOS' consultants. Standardization and consistency of presentation will be reinforced during the training session. The culmination of this training will be a mock assessment center that will combine the assessors and role-players. Assessors: Prior to the administration of the assessment centers, assessors will receive an in- depth training session that focuses on a thorough description of the target rank and its responsibilities within the department, the assessor's role, the dimensions that are measured for the exercise with which the assessor will be involved, careful note-taking, behavioral observation, scoring, common rating errors, reliability of assessment, providing candidate feedback comments and specific assessor duties. Ideally, a representative of the department can be present to answer any specific questions that may arise. This training will also allow ample time for assessor questions and for practice with mock candidates. This training session will end with a mock assessment center that involves the role-players. 118 Copyright©2015,Industriol/Orgonizotionol Solutions,Inc CONFIDENTIAL—TRADE SECRET Assessment Administration IOS oversees the administration of the assessment center process. We send sufficient personnel to conduct all essential functions of orienting candidates, moving candidates throughout the assessment process, answering candidate questions, etc. We may request minor administrative support from the agency to assist in candidate sign-in or to monitor candidates. Prior to mapping out a final assessment administration plan, IOS will meet with agency personnel to discuss administration options. There are several competing interests in any assessment center administration process including: assessment length, the type of exercises utilized, test security and other related factors. Depending on how your agency views these competing interests, a decision must be made regarding how to implement your assessment center administration. IOS has implemented assessments using each of the methods described below on numerous occasions and is intimately familiar with the nuisances of each method. There are typically two contrasting methods—a sequestering method and a traditional model. Traditional Model of Assessment Administration In order to have the most robust and convenient testing process, it is usually ideal to use one assessor panel to rate each exercise (every candidate is seen by the same assessors per exercise). That said, this requires multiple days of administration, requires the use of signed confidentiality agreements and involves an element of "trust" in regard to your candidate population. This typifies the "traditional" model of assessment administration. Since we can typically process a maximum of 16-18 candidates per day in a given rating panel, the number of days is then determined by the number of candidates divided by this number (i.e., I S or 18). For example, assuming that there are 48 candidates scheduled to participate in an assessment center, with three exercises and three assessors per rating panel. This example would require three rating panels (nine total assessors if using three per panel) and three (3) days of assessment administration and an additional day for training. While this system allows for many advantages (i.e., fewer assessors required, less staffing burden on the agency, greater standardization of rating panels, etc.), the one major concern is that this system allows for a potential breach of security should a candidate from day one inform other candidates in subsequent days of the content of the assessment. Certainly, this concern can be preempted by requiring candidate confidentiality agreements, but these are not a perfect protection. Therefore, we seek to inform agencies of the strengths and weakness of various models and select the system that best meets the needs and concerns of the agency. An alternate option is the use of a sequestered assessment model. Sequestered Assessment Administration Model(s) In a sequestered assessment, all candidates compete in a given exercise in a single day. A sequestering approach may utilize an a.m. arrival group and a p.m. arrival group, All candidates are sequestered in the process for 4-6 hours to ensure there are no security leaks. There are numerous variations on this approach. In the most typical version of this approach, all candidates are processed through all exercises in a single day. This approach becomes challenging with a large number of candidates but is entirely workable. For example, if there are 48 candidates scheduled for an assessment and two live exercises (with a third written 119 Copyngtt 0 2015,Industrial/Organizationol Solutions,Inc. CONFIDENTIAL—TRADE SECRET exercise), a city would require six (6) rating panels to run the assessment and a large number of rooms. All exercises could be administered in a single day and the two live exercises could be evaluated on the first day. The written exercise could then be evaluated on a second day. If a third live exercise was introduced instead of a written exercise, nine (9) rating panels would be required in this example, assuming that 16 candidates could be processed by a panel in a single day. Another variation on this approach is to require multiple days of participation by candidates. IOS has utilized this process with the Houston Police Department, the Atlanta Police Department, Washington D.C. Fire & EMS and the Buffalo Police Department. In this approach, candidate sequestering is again utilized, though candidates are required to attend the assessment across two or three days. This method can cut down on the number of rating panels needed as all candidates do not need to be processed in a given exercise in a single day. The city of Houston employed this model with a 50-candidate lieutenant assessment center in 2010. IOS required participation by candidates across three consecutive days. A sequestering process was used within each day to ensure absolute test security. Candidates were "held" within the assessment for 3-6 hours each day. This process was accomplished in three days, while a fourth day was reserved for training. The disadvantage of this approach is that it is not as convenient for those competing in this process. The advantage of this approach is that a sequestered process can be utilized with only three rating panels. A third variation employs the use of videotaping. While other vendors use this method as a primary assessment administration method, we have utilized this method on occasion to address a concern or solve a logistical issue involving the administration (or when required by contract). An advantage to this model is that the videotaping methodology allows the administrator to process more candidates in a day (upwards of 25 per room vs. 15-18) and allows for fewer rating panels to evaluate the exercise. With this method, the administration can be conducted as a sequestered process, with the ratings occurring over a longer period of time. A disadvantage to this model is that candidates far prefer interaction with live assessors than giving a presentation to a video-camera and they are less likely to rate the process as highly job-related. In some instances, we have combined a videotaping approach with a multiple-day sequestering approach to gain further efficiencies. In summary, IOS and our staff are experts in assessment center implementation. We will evaluate your situation and your preferences and recommend an appropriate approach. We have a great deal of experience with each approach and can work effectively with all methods. IOS will produce all exercise materials needed for the administration of the assessment center and will develop an administration schedule that allows for the efficient processing of all eligible candidates through the assessment center in the most secure manner possible. IOS will work with the department to ensure that the administration schedule is effective based on personnel staffing and manpower concerns and any specific concerns that the department may have about test security. IOS will provide all equipment necessary to record each candidate's performance 120 Copyright U 2015,IndustriollOrgonizotionol Solutions,Inc CONFIDENTIAL—TRADE SECRET for each exercise, as well as the rating portion. These recordings will be made available to the City. While IOS will oversee the administration of the assessment center, candidates will actually be rated by assessors from outside agencies that hold a rank at or above the level of this target process. One of IOS primary jobs is to monitor assessor performance and to ensure standardized and reliable ratings are being obtained. Candidate Feedback IOS proposes to provide individual feedback reports to each candidate and to make available video of the live exercises for review by each candidates. We recommend that individual candidate meetings to discuss performance not be conducted. While we are willing to conduct these meetings if absolutely necessary, our substantial experience in this industry leads us to conclude the thorough feedback reports are a far superior solution. During the assessment center, assessors will be required to document the candidates' primary strengths and developmental needs for each assessment exercise. During assessor training, assessors will be instructed as to what information to include as feedback. Assessors will also be encouraged to highlight points for each assessment exercise that increased the candidates' performance for that scenario or decreased the candidates' performance. All comments will be screened to ensure that assessors are identifying behaviors that will be understandable to the candidate. This screening will also ensure that there are no inappropriate comments included in the candidates' feedback reports. Candidates will also be offered normative information so that they can compare their performance with that of the rest of their peer group. This information will assist the candidates in understanding the scale and how other candidates performed as a group. 121 Copyright©2015,fndusuiollOrgonizotionol Solutions,lnC. CONFIDENTIAL—TRADE SECRET • 4t Project Timeline • Sergeant Promotional Process Project Component Timeframe Kick-off meeting/JA review Week I Written exam development _ Weeks 2-9 Candidate orientation Week 5 Written exam administration Week 10 Assessment center design Weeks 5-1 I Assessment center administration Week 12 Scoring, reporting and generation of feedback reports Week 13 122 Copyright Q 2015,IndustriollOrganaotionol Solutions,Inc. CONFIDENTIAL—TRADE SECRET