How to Evaluate the Quality of Technical Credentials

02.22.2021 | Credentialing

Within the world of career and technical education (CTE), the word “credential” is linked to thousands of options promoting technical competence. But there is so much more to a credential than simply creating a test that is marketed as measuring a learner’s skills and knowledge. Here are some considerations to use as you take on the task of determining the value and quality of technical credentials.

What Does “Credential” Mean to You?

It is difficult to evaluate a credential’s quality without first understanding the full scope of what “credential” can mean. Generally within CTE, “credential” is a catchall term signifying the assessment of a learner’s technical competence. However, the term can have slightly different applications from one organization to the next.

For example, Credential Engine, a nonprofit organization working to standardize the language used for credentials, has listed the various subclassifications of “credential”:

  • Badge
  • Certificate
  • Certificate of Completion
  • Certification
  • Degree
  • Diploma
  • License
  • Micro-Credential
  • Quality Assurance Credential

Credentials also have various applications depending on how learners, teachers, administrators, and employers use them and interpret their intent and value.

For Learners

For those learning new skills, a credential is a way to measure technical competence. Preparing for the assessment portion of a credential helps students focus on specific areas of study. A completed credential is tangible proof of skills and knowledge.

For Teachers & Administrators

For teachers and administrators, a credential should provide confidence that your program has equipped learners with industry-relevant skills. Data from the credentialing process should also give you the insight to provide individualized instruction for students, drive program improvement, and target professional development needs.

For Employers

For employers, a credential is an assurance that the candidate has industry-relevant skills and knowledge. This helps them find qualified employees, structure training programs, and work towards building a skilled and competent workforce.

Beneath the Surface

Simply evaluating the assessment or test won’t be enough to determine the full value of the credential. Every aspect—from the credentialing organization to the assessment development process to data analysis and reporting—should be examined.

Here are some points to consider before choosing a credential provider for your program.

The Provider

  • Is the credential provider accredited and do they follow international standards? The International Certification Accreditation Council (ICAC) is an accrediting body that confirms organizations are meeting the standards of ISO 17024. ISO 17024 standards are published by the ISO Committee on conformity assessment (CASCO) and are reviewed every five years.
  • Does the credential provider follow the Standards for Educational and Psychological Testing developed jointly by the American Education Association (AERA), the American Psychological Association (APA), and the National Council of Measurement in Education (NCME)?
  • Do third-party organizations place value on the provider’s credentials?
  • Who uses the provider’s credentials?
  • Does the credential match the content delivered in your CTE program?

The Credentialing Assessment

  • Who will be taking the credentialing assessment?
  • Did the credentialing assessment follow industry guidelines during development?
  • Does the credentialing assessment measure skills determined by a national association (e.g., American Culinary Federation) or is it for a private business product (e.g., Microsoft, Google)?
  • Who were the subject matter experts (SME) involved? Who was responsible for vetting the SMEs’ qualifications?
  • Is there a technical manual available?

The Data

  • What comparative credentialing assessment data is available from the credential provider? Are the reports broken out showing individual skills and competencies?
  • How timely can you receive the results? Data is a critical part of planning for the next education cycle.
  • Do the reports make it easy for you to spot trends and patterns?
  • Do the reports include comparative data such as state and national averages and pre- and post-test comparisons?
  • It’s helpful to look at scores available by student, by class, and by program. Look for patterns and determine strategies for instructional improvement. Unless you have the data, you really don’t know.

Exploring Data Derived from Credentials

“Without data you’re just another person with an opinion.” W. Edwards Deming, Data Scientist

NOCTI has over 55 years of experience with credentialing programs. Not only do we provide technical credentials, but we also have a number of tools and resources designed to help you meet Perkins V requirements and use educational data for instructional improvement.

For example, one of the most significant changes to Perkins V is a requirement for a Comprehensive Local Needs Assessment (CLNA). This process involves reviewing a number of elements, including student performance data, progress toward implementation of CTE programs, and more. The CLNA must be completed by local recipients of Perkins funds at the beginning of the grant period and then updated at least once every two years.

Here’s how NOCTI’s services can assist with CLNA requirements:

Performance on federal accountability indicators.

Applicable NOCTI services:

  • Industry credentialing; pre-tests also available
  • Competency and data analysis
  • Professional development
  • Subpopulation data analysis

Size, scope, and quality of programs offered.

Applicable NOCTI services:

  • Foundational credentials
  • Micro-credentials
  • College credit recommendations

Recruitment, retention, and training of faculty and staff.

Applicable NOCTI services:

  • Teacher assessments
  • CTE teacher professional development
  • CTE administrator professional development
  • Prior learning assessments

Progress toward improving access and equity.

Applicable NOCTI services:

  • Micro-credentials
  • Subpopulation analysis

About NOCTI

NOCTI has earned accreditation from the ICAC and follows the standards and best practices for credentialing outlined in ISO 17024. With over five decades of experience in CTE, our comprehensive services, in-depth reporting, and industry resources help our credentialing partners meet regulatory requirements while also enabling data-driven program improvements and student success.

In addition to our services, NOCTI’s connections and credibility within the education community create ample opportunities for dialogue with key strategic partners. NOCTI is an active member of the Association of Test Publishers (ATP). ATP members pledge to promote and advance the integrity of assessment services and products and their value to society and dedicated to the highest level of professionalism and business ethics within the test publishing community.

To learn more about determining the quality of technical credentials, watch a NOCTI Spotlight Series Webinar led by President/CEO Dr. John Foster.

Other resources you may be interested in: