## Overview The **Psychometric Analysis Tool** is a sophisticated statistical analysis component within WASPL that evaluates the quality and reliability of educational assessments. It provides comprehensive psychometric analysis capabilities for educators and researchers to validate their test instruments according to professional measurement standards. ### 📊 Statistical Analysis Comprehensive reliability analysis using Cronbach's Alpha, item discrimination, difficulty analysis, and item-total correlations. ### 🎯 Quality Assessment Automated quality indicators with professional thresholds and recommendations for test improvement. ### 📋 Multi-Publication Analysis Compare multiple test administrations or combine data for robust statistical analysis. ### 🔍 Data Validation Built-in detection of methodological issues, outliers, and data quality problems.

## Getting Started

### Access the Tool Navigate to your test in WASPL Editor and select the **Psychometrics** tab. Only tests with EXAM mode publications will show analysis options.

### Review Publications The tool automatically loads all eligible publications. Review the summary statistics and quality indicators for each publication.

### Select Data Choose which publications to include in your analysis. Use quick selection tools or manual selection based on your research needs.

### Configure Analysis Select analysis type (Individual, Grouped, or Comparative) and configure data preprocessing options.

### Run Analysis Execute the psychometric analysis and review the comprehensive results with recommendations.

### Export Results Generate professional reports in PDF format or export raw data for further analysis. ### 💡 Prerequisites

- **EXAM Mode Publications**: Only publications in EXAM mode are eligible for psychometric analysis - **Minimum Sample Size**: At least 10 participants recommended for basic analysis - **Complete Responses**: Best results require high completion rates (80%+)

## Publication Selection ### Understanding Publication Cards Each publication is displayed with comprehensive information to help you make informed selection decisions:

👥

Participant Count

Total number of students who attempted the test

✅

Completion Rate

Percentage of students who completed all items

⏱️

Average Time

Mean completion time for the assessment

🔍

Data Quality

Automated detection of anomalies or issues ### Quick Selection Tools #### ☑️ Select All Include all available publications for maximum sample size #### 🕐 Most Recent Select the 3 most recent publications for current performance analysis #### 📈 Largest Samples Choose publications with the highest participant counts for statistical power ### Filtering and Sorting - **Search Filter**: Find publications by name or keyword - **Sort Options**: Order by date, participant count, completion rate, or alphabetically - **Minimum Participants**: Set threshold to filter out small samples ### ⚠️ Sample Size Recommendations

- **N ≥ 100**: Required for robust IRT analysis and factor analysis - **N ≥ 50**: Minimum for exploratory factor analysis - **N ≥ 30**: Sufficient for reliable Cronbach's Alpha estimates - **N < 30**: Limited to basic descriptive statistics

## Analysis Types ### 🔬 Individual Analysis **Purpose**: Analyze each publication separately for comparison **Use Case**: Compare performance across different administrations, groups, or time periods **Output**: Separate reliability and item statistics for each publication ### 📊 Grouped Analysis **Purpose**: Combine all selected publications into one comprehensive analysis **Use Case**: Maximize sample size for robust statistical estimates **Output**: Single set of psychometric statistics based on combined data ### 🔀 Comparative Analysis **Purpose**: Global analysis plus between-group comparisons **Use Case**: Research studies comparing different populations or conditions **Output**: Combined statistics plus significance tests between groups ### 💡 Recommendation **Grouped Analysis** is recommended for most educational applications as it provides the most reliable statistical estimates by maximizing sample size. Use Individual Analysis when you need to compare specific administrations or investigate changes over time.

## Quality Indicators & Thresholds ### Reliability Categories (Cronbach's Alpha) #### A - Excellent α ≥ 0.90

Outstanding reliability for high-stakes testing

#### B - Good 0.80 ≤ α < 0.90

Good reliability for most educational purposes

#### C - Acceptable 0.70 ≤ α < 0.80

Acceptable for formative assessment

#### D - Poor α < 0.70

Needs improvement before use

### Item Quality Standards

Metric	Good	Acceptable	Problematic	Interpretation
Difficulty	30-70%	20-80%	<20% or >80%	Percentage of students who answered correctly
Discrimination	≥0.40	0.30-0.39	<0.30	Ability to distinguish high from low performers
Item-Total Correlation	≥0.30	0.20-0.29	<0.20	Consistency with overall test performance
Point-Biserial	≥0.25	0.15-0.24	<0.15	Alternative discrimination measure

### 🎯 Quality Interpretation

- **Green Items**: Meet or exceed quality standards - retain these items
- **Yellow Items**: Acceptable quality but could be improved
- **Red Items**: Below standards - consider revision or removal

## Data Preprocessing ### Methodological Issue Detection The tool automatically identifies common methodological issues that can affect analysis validity: ### 🔄 Multiple Attempts **Issue**: Students taking the test multiple times **Impact**: Learning effects, violation of independence **Solution**: Use only first attempts or best attempts ### ⚠️ Incomplete Data **Issue**: Students who didn't complete the test **Impact**: Selection bias, reduced statistical power **Solution**: Exclude incomplete responses or use imputation ### 📈 Sample Size **Issue**: Insufficient sample size for chosen analysis **Impact**: Unreliable estimates, reduced power **Solution**: Combine publications or limit analysis scope ### ⏱️ Timing Anomalies **Issue**: Extremely fast or slow completion times **Impact**: Invalid response patterns **Solution**: Automatic outlier detection and exclusion ### Quality Control Options - **Multiple Attempts Exclusion**: Automatically keep only first attempts - **Completion Threshold**: Set minimum percentage of items completed - **Timing Filters**: Remove responses with suspicious timing patterns - **Response Pattern Analysis**: Detect random or non-engaged responding ### ⚠️ Statistical Assumptions Psychometric analysis assumes:

- Independence of observations (no collaboration) - Unidimensional measurement (items measure the same construct) - Sufficient sample size for stable estimates - Honest responding (students trying their best)

## Interpreting Results ### Overall Test Quality The analysis provides an overall grade (A-D) based on multiple quality indicators: #### 📊 Analysis Results Overview **Overall Grade:** B (Good Quality) **Cronbach's Alpha:** 0.84 (Good Reliability) **Sample Size:** 156 participants **Items Analysis:** 12 Good, 6 Acceptable, 2 Problematic ### Item-Level Analysis Each test item receives detailed statistical analysis:

Item	Difficulty	Discrimination	Item-Total r	Status	Recommendation
Item 1	65%	0.45	0.42	✓ Good	Retain - excellent quality
Item 2	35%	0.32	0.28	⚠ Acceptable	Consider slight revision
Item 3	15%	0.18	0.12	✗ Problematic	Review or remove - too difficult

### Recommendations #### ✅ Actions for Test Improvement

- **Retain high-quality items** (discrimination ≥ 0.40) - **Revise problematic items** with low discrimination or extreme difficulty - **Consider removing items** that don't contribute to test reliability - **Add more items** if overall reliability is below 0.80

## Best Practices ### Sample Size Guidelines #### 🎯 For Classroom Assessment

- Minimum N = 20 for basic reliability - Target N = 30+ for stable estimates - Combine classes when possible

#### 🔬 For Research Studies

- Minimum N = 100 for IRT analysis - Target N = 200+ for complex models - Power analysis for group comparisons

#### 📊 For High-Stakes Testing

- Target N = 500+ for operational use - Multiple field test administrations - Cross-validation with independent samples

### Data Quality Checklist #### ✓ Before Running Analysis

- Verify test was administered under standardized conditions
- Check for adequate completion rates (>80% recommended)
- Review timing data for suspicious patterns
- Ensure sample represents intended population
- Document any special circumstances during administration

### Interpreting Low Reliability #### 🔍 Common Causes of Poor Reliability

- **Too few items**: Reliability increases with test length - **Heterogeneous content**: Items measuring different constructs - **Poor item quality**: Items with low discrimination - **Inappropriate difficulty**: Items too easy or too hard - **Small sample size**: Unstable estimates with N < 30

## Troubleshooting ### Common Issues and Solutions ### ❌ No Publications Available **Cause**: Only EXAM mode publications are eligible **Solution**: Ensure test has been published in EXAM mode with student data ### ⚠️ Analysis Fails **Cause**: Insufficient data or computational error **Solution**: Check sample size, data completeness, and try simpler analysis ### 📊 Unrealistic Results **Cause**: Data quality issues or methodological problems **Solution**: Review preprocessing options and data collection procedures ### 🐌 Slow Performance **Cause**: Large datasets or complex analysis **Solution**: Reduce sample size or simplify analysis type ### Error Messages

Error	Meaning	Solution
"Insufficient data"	Sample size too small	Select more publications or reduce analysis complexity
"No variance in responses"	All students gave same answers	Check item difficulty and administration conditions
"Matrix not positive definite"	Correlation matrix issues	Remove problematic items or increase sample size
"Analysis timeout"	Computation took too long	Reduce sample size or contact support

## Technical Details ### Statistical Methods

Metric	Formula/Method	Purpose
Cronbach's Alpha	α = (k/(k-1)) × (1 - Σσᵢ²/σₓ²)	Internal consistency reliability
Item Difficulty	p = Number correct / Total attempts	Proportion of students answering correctly
Item Discrimination	Point-biserial correlation	Ability to differentiate performance levels
Item-Total Correlation	Corrected correlation (item removed from total)	Consistency with overall performance

### Computational Features - **Missing Data Handling**: Listwise deletion or pairwise correlations - **Outlier Detection**: Z-score and timing-based filtering - **Bootstrap Confidence Intervals**: For reliability estimates - **Effect Size Calculations**: Cohen's d for group comparisons ### Export Formats #### 📄 PDF Report Professional formatted report with all statistics, charts, and recommendations #### 📊 JSON Data Raw statistical output for integration with other tools or custom analysis #### 📈 CSV Export Item-level statistics for spreadsheet analysis or graphing ### 🔧 Integration with WASPL

- **Test Repository**: Pulls item information and test structure - **Results Database**: Accesses student response data - **User Authentication**: Integrated with WASPL security system - **Publication System**: Links to test administration records