CONTENT validity - correct answer the assessment measures the content it's designed to assess
Purpose of CONSTRUCT validity - correct answer Checks to
... [Show More] see if test measures what it's supposed to and not something else
Purpose of CONTENT validity in objective assessment - correct answer To determine how closely the test represents what should be assessed.
Objective assessment - correct answer Question with only one answer
Purpose of CRITERION-RELATED validity - correct answer Measures the degree of correlation between a first and 2nd test
CONSEQUENTIAL validity - correct answer results are used for a purpose and have social significance.
FACE validity - correct answer looks valid on the surface
CONSTRUCT validity - correct answer Degree to which a test actually measures what it claims to measure
CRITERION-RELATED validity - correct answer Validity is determined by comparing one test to another (2 types: concurrent & predictive)
PREDICTIVE validity - correct answer results used to determine future achievement
CONCURRENT validity - correct answer the extent to which two measures of the same trait or ability agree
OR
how well a new test compares to a well-established test
How do you ensure validity of a test? - correct answer 1. Scoring procedures are evaluated for objectivity FAIRNESS
2. Test is evaluated for evidence bias
CHECK for BIAS
3. test is evaluated for content consistency
CONSISTENT
Threats to validity - correct answer 1. Ambiguous test directions
2. cheating by students
3. subjective scoring methods
INTERNAL reliability - correct answer consistent results across items in a test
SPLIT HALF = both halves of the test "work"
Kuder-Richardson
Chronbach's alpha
estimated from a single administration of a test
check for sufficient # of items to assess reliability
EQUIVALENCY reliability - correct answer the extent in which 2 or more forms of a test are consistent (version A and version B)
INTER-RATER reliability - correct answer A measure of how similarly two different test scorers would score a test.
Consistency of a scoring system
STABILITY reliability - correct answer Measurement reliability across time;
a measure that yields consistent results at different time points assuming what is being measured does not itself change
STABILITY Reliability - correct answer Test-Retest reliability
internal reliability - correct answer split-test reliability
Reliability is essential because... - correct answer shows ability to perform and maintain effectiveness
Methods that determine reliability... - correct answer 1. Inter-Rater
2. Parallel Forms
3. Split-half measure
4. Chronbach's alpha
5. Kuder-Richardson
High RELIABILITY coefficients show... - correct answer highly reliable
Low RELIABILITY coefficients show... - correct answer low reliability
Characteristics of reliability coefficients measure... - correct answer 1. how well scores hold up over time
2. the degree to which the items on a test are homogeneous
3. the closeness of the judges' scores
Constructed Response - correct answer a written response to a prompt (usually a paragraph)
LOW reliability (depending on inter-rater folks)
Projective Test - correct answer a personality test, such as the Rorschach or TAT,
LOW reliability
Heterogenous test group - correct answer HIGH reliability
Large number of test questions - correct answer HIGH reliability
a short test, a quiz - correct answer LOW reliability
Significant difference in students' scores - correct answer LOW reliability
Several subtest components of one larger test - correct answer LOW reliability
negligible (insignificant) standard error of measurement - correct answer typical error of measurement is insignificant
HIGH reliability
Significant standard error of measurement - correct answer measure of how much measured test scores are spread around a "true" score.
more errors = the less accurate the test
LOW reliability
Minor differences in students' scores - correct answer HIGH reliability
Inconsistent rating of constructed response items - correct answer LOW reliability
Spearman-Brown used for VALIDITY or RELIABILITY? - correct answer RELIABILITY
FORMATIVE - correct answer never graded, anonymous
used to guide instruction
review of the interim & final project deliverable by development team (checking on if they have the right goals)
review by internal & external designers to improve navigation (software)
Performance-based assessment - correct answer An alternative assessment method based on a student's performance of a skill based on a real-life situation.
SUMMATIVE - correct answer end-of-course test
ie: survey to measure student satisfaction or achieve unit goals or multiple guess
IMPACT evaluation - whether an intervention is effective or not
Placement Assessment - correct answer (measures entry behavior) To determine student performance at the beginning of instruction
Example: Unit Pre-test
purpose of formative assessment - correct answer to improve CURRICULUM not students
Implementation Evaluation - correct answer documents the evolution of a project and provide indications of what happens within a project and why it happens.
Project directors use information to adjust current activities.
FORMATIVE evaluation element
An ELEMENT of Formative Evaluation - correct answer IMPLEMENTATION evaluation
(notes the changes needed as instruction progresses)
purpose of summative evaluation - correct answer To assign grades or certify mastery
see if goals have been met
make decisions
Identify subject of the evaluation - component of... - correct answer SUMMATIVE Evaluation
Define the purpose of an evaluation - component of... - correct answer SUMMATIVE Evaluation
Who are stakeholders? Plan the eval - components of... - correct answer SUMMATIVE Evaluation
determine the data needed for answers - component of... - correct answer SUMMATIVE Evaluation
Decide where data is, how to collect it, collection instruments - components of... - correct answer SUMMATIVE Evaluation
Write the narrative timeline of when & how data will be collected - component of... - correct answer SUMMATIVE Evaluation
SELECTED RESPONSE test - correct answer choose answer
easier to grade
higher reliability (WGU tests)
CONSTRUCTED RESPONSE test - correct answer short answer, essay or other open ended questions
pain to grade
lower reliability
Components of CONSTRUCTED response: - correct answer clear directions
ample space for answers
give details about what is required in answer
INTERVIEW - correct answer A face-to-face or telephone questioning of a respondent to obtain desired information.
questionaire - correct answer Provide answers to well-defined questions
electronic format
provide anonymity
qualitative data - correct answer descriptive data
discrete
nominal (doesn't overlap - male/female...)
Dichotomous Scoring - correct answer Only two events are possible
(i.e. heads vs tails)
QUANTITATIVE data
Polytomous Scoring - correct answer More than two possible answers
QUANTITATIVE data
discrete variable - correct answer a quantitative variable that has either a finite number of possible values
or a countable number of possible values
Coding/Indexing - correct answer technique for quantifying QUALITATIVE data
Essay exam - QUALITATIVE or QUANTITATIVE? - correct answer Qualitative :o
Closed end questions - QUALITATIVE or QUANTITATIVE? - correct answer QUANTITATIVE ###
Likert Scales 1-5 - QUALITATIVE or QUANTITATIVE? - correct answer QUANTITATIVE ####
Comparison of Pre-test vs post test scores
QUALITATIVE or QUANTITATIVE? - correct answer QUANTITATIVE ####
Summative evaluation using student test scores
QUALITATIVE or QUANTITATIVE? - correct answer QUANTITATIVE ###
Semantic differential scale on a continuum
QUALITATIVE or QUANTITATIVE? - correct answer QUANTITATIVE ####
Interview to capture scaled responses
QUALITATIVE or QUANTITATIVE? - correct answer QUANTITATIVE (scaled - #s) [Show Less]