A Bounty of Metrics, A Bafflegab of Measures and Targets

Metrics and Targets

One of my motivations for compiling this information was to highlight that even the twenty-one system-wide metrics, did not have system-wide methods for measuring targets. Practically, each university has been able to provide public updates based on these measures for their first-year SMA report (Brock University, University of Toronto, Western University) but in many cases, it would be very difficult for anyone other than the university itself to measure the system-wide metric against the agreed target. This is before we consider the 244 institutional targets and metrics, of which 228 were unique.

The data powering the dashboard available in the Almost All SMA2 Metrics & Targets In a Dashboard post is based on these metrics. Power BI allows for individuals to compare specific universities to each other or the system, it also allows for comparing multiple dimensions of data, the last slide (12) lists all 228 unique institutional metrics – but does not include the measures. All system-wide and institutional metrics and measures, both literal and interpreted are available in the spread available in that post (remove the Scope filter to find the institutional metrics and measures).

Many targets set for system-wide and institutional-metrics did not have immediate utility in measuring the target against other institutions or even the institution itself.  Targets that were set relative to private or internal baselines, or relative to regional or provincial averages were particularly challenging.  This table depicts the first metric and is representative of the targets set for many other metrics and how I interpreted them to make the comparative Dashboard.

Proportion of fourth year students with two or more High Impact Practices

University
Literal Target
Target as Interpreted
Explanation
AlgomaMaintain at current Ontario average of 54%54%Percentage provided was used.
BrockMaintain or improve current proportion (53 %)53%Lowest & most specific target used
LaurentianIncrease from 62.3% on the National Survey of Student Engagement (NSSE) 2014 (last year available) to 63% on NSSE 2020 the %age of fourth-year students who participated in 2 or more High-Impact Practices. Includes participation in a learning community, service learning, research with faculty, participation in an internship or field experience, study abroad, and culminating senior experience.63%Ultimate target used
Nipissing
(Similar targets were set by Lakehead, Queen’s, Ryerson, UOIT/OUT & York)
Target Range: 52-58%55%Midpoint used
Guelph
(Similar targets were set by Trent & Waterloo)
More than 50% of fourth-year students will have 2 or more HIPs (as measured by NSSE).50.1%Added a tenth of a percent to be more than 50%
Ottawa
(Similar targets were set by Wilfrid Laurier)
53% or 1.78 HIPs per student53%Percentage measure was used
Toronto≥ Ontario mean55%Mean of actuals not available when calculated, mean of targets used (but one could find the actuals in 2020)
University of WindsorStrive to ensure that a minimum of 55% of our fourth year students have approximately two HIPs55%The minimum was recorded as the lowest & most specific
Western University
(Similar targets were set by Hearst, Carleton, McMaster & OCADu)
50%50%Literal target was functional for comparison
Many targets set for system-wide and institutional-metrics did not have immediate utility in measuring the target against other institutions or even the institution itself. Targets set relative to private or internal baselines, or relative to regional or provincial averages were particularly challenging. This table depicts the first metric, ID 34 in annual reporting, and is representative of the targets set for many other metrics.