Job Summary:
The Director of Test Development provides leadership and direction for the Test Development team. This position is crucial for ensuring the accurate and timely development of over 50 high-stakes examinations and longitudinal assessments annually. The role involves significant independence, judgment, and initiative to ensure project completion with a focus on timeliness and accuracy.
Essential Duties and Responsibilities:
- Oversee all aspects of the test development processes and assessment products while integrating best practices, current research, and innovations in testing within the department.
- Provide test development expertise to staff and subject matter experts (SMEs), while building and maintaining the necessary strong relationships to achieve desired results.
- Oversee recruitment for the Certification Development Experts, General Pediatrics Examination Committee, and General Pediatrics MOCA-Peds Committee, as well as Medical Editors for the Subboards.
- Design test development processes, workflows, and committee/subboard meetings to ensure high-quality exams and assessments.
- Manage the team by setting clear objectives, establishing priorities, tracking projects, and delegating responsibilities appropriately.
- Promote a culture of collaboration, particularly between the test development team and the entire organization, particularly the psychometrics team, to strengthen integration and cooperation to drive assessment forward.
- Coordinate and partner with vendors to ensure the development and delivery of high-quality assessments.
- Explore, plan, lead, and implement innovative approaches to testing and item development (eg AI for item generation), including but not limited to process improvements and strategic initiatives.
Other Functions:
- Support the training and orientation of SMEs.
- Represent the ABP at national professional organizations and serve as a testing expert on various internal committees.
- Communicate effectively with external stakeholders, including the American Academy of Pediatrics (AAP), American Board of Medical Specialties (ABMS), other member boards, diplomates, and the wider testing community.
- Engage the department in research activities and share results with national testing audiences.
- Oversee the creation and maintenance of standard policy and procedure documents relating to test development activities ensuring compliance with industry standards.
- Initiate, lead, and participate in special projects outside of standard role and responsibilities.
- Performs other duties as assigned.
Supervisory Responsibilities:
This role currently supervises a team of 8 members.
Required Qualifications:
- Minimum of a master's degree in Educational Measurement, Educational Statistics, Industrial/Organizational Psychology, Education, Psychology, or a related field. Additional experience may be considered in lieu of the educational requirement
- Minimum of 7 years of related experience in assessment and testing, particularly certification
- At least 5 years of personnel management experience
- Demonstrated experience in managing vendors
Preferred Qualifications:
· Excellent leadership, motivational, and organizational skills
- Strong communication skills, both written and in-person
· Solid time and task management skills
· Strong analytical skills: ability to solve complex problems
- Working knowledge of LLMs and AI.
· Strong innovative skills, both in conceptualization and implementation
· Ability to integrate technological improvements and solutions
· Proficiency with item-banking best practices and software (eg, ExamDeveloper)
· Expert knowledge of computer and Internet-based testing
· Proficient with project management tools and applications
· Expertise in Microsoft Office applications, particularly Excel and SharePoint
· Superior communication skills, both oral and written
· Demonstrated public speaking and presentation skills
· Excellent facilitation skills