Tech metrics beyond surveys

 

Our district is finalizing its strategic roadmap for the next three years. Each of the directors - curriculum, special education, human resources, community ed, etc. - has been tasked with creating a "vision card" that outlines the core services of his/her department and identify metrics to determine whether those services are being delivered.

So guess what I am working on. Yes, tech gets its own card.

Our department has for many years defined its "core services" in the language of long-term goals. We've built annual short term objectives that support the long-term goal and track and report how many of these objectives we meet each year. Here's an example: 2013-14

The purpose of the vision card, as I understand it, is to assign specific metrics to evaluate the degree to which service is being met. So, OK. Here are some of my initial ideas...

Goal/Core Service #1: All students will demonstrate proficient use of technology to access, process, organize, communicate and evaluate information in order to answer questions and solve problems and to practice digital citizenship.

Measure: Percent of students who demonstrate mastery of identified technology-related skills through performance assessment. (Elementary level easy to assess; secondary, not so much.)

Vision: 100% of students at all grade levels demonstrate proficiency in information literacy and technology skills

Goal/Core Service #2: Technology will be used to provide the most current, accurate and extensive information and learning resources possible to all learners in the district and community in a cost effective and reliable manner at maximum convenience to the user. 

Measure: Percentage of students who have access to a computing device when needed.

Vision: There is a 1:1 ratio of students to devices supplied by the school 24 hours a day.

Goal/Core Service #3: All district teachers will have the technology training, skills and resources needed to assure students will meet local and state learning objectives and have the technological means to assess and record student progress. 

Measure: Percentage of teachers who participate in professional development activities with technology components (Do we want % of teachers having a tech IEP? Should this be moved to PD area?)

Vision: 95% participation in technology-related professional development activities that result in changed practice within 14 days

Measure: Percentage of teachers who use technology effectively in the classroom (Use “Look-for’s” in Danielson model to gauge.)

Vision: 95% of teachers effectively use of technology on a weekly basis

Goal/Core Service #4: The district will use technology to improve its administrative effectiveness through efficient communication, planning, and record keeping. 

Measure: Percent of families satisfied with ability to access their student’s information in real time.

Vision: More than 90% of families use student information system parent portal and report satisfaction with the information via survey

Goal/Core Service #5: The district will have a reliable, cost-effective, and secure technology infrastructure that supports the learning, teaching, and administrative goals of the district. 

Measure: Percentage of time digital resources are available to users in a reliable, adequate, and secure manner.

Vision: Less than .0001 unscheduled downtime and no reported security breaches. Bandwidth meeting 100% of COSN standards (2017-18)  of 10 gig per 1000 users.

If you've stuck with this this long, I'm guessing you may have done something similar in the past or want to do something similar in the future. So share your ideas. How do you measure the effectiveness of your tech department's services? And what's your vision for accomplishing them?

Comments

[Darren Draper]

I love that you've shared this, Doug, and hope to share a similar set of goals soon!

A few questions:

1. How do you assess your student technical proficiency (goal 1)? Do you use some sort of standardized test?

2. Would you be willing to share more about your "look fors" related to goal 3? How are you assessing the quality of the instructional use of technology?

[Joel VerDuin]

Doug - some of your colleagues have developed service level agreements about most of the "services" they provide as a central department. Those might make a natural fit for performance indicators to say what is good, better or best. Edina and Eden Prairie presented on this topic so Steve Buettner or Josh Swanson might be really helpful.

Secondly, I had seen some metrics that caught my eye and I also have been mulling over for our own performance indicators. You might find something valuable in here.

We had a BYOD statistic about number of devices - but then I thought it might be more telling if we asked teachers or students about use.

I have, in the past, used metrics around completing support calls - but am really intrigued about the idea of reducing the number of calls (not how fast we turn them around).

For professional development, I've been considering measuring what eventually lands in classroom practice as opposed to how much professional development occurs. I am not fully sure how that might look yet, but it is still interesting to ask if we do things to be more impactful on practice.

Just a couple of thoughts.

[Scott McLeod]

Good stuff, Doug. Thanks for sharing.

My team is trying to move its outcomes thinking from teacher activities to student work products. In other words, if we showed teachers how to do X, our ultimate measure of success is not that teachers participated in X training but rather that we actually see students' X-related work products...

[Doug Johnson]

Hi Darren,

Student proficiency at the elementary level we can determine based on whether kids meet ELOs in their library/media classes, I believe. At the secondary level, I am proposing this plan:

<https://docs.google.com/document/d/1a0pi-xdfAoGucm9XItzzq-q8zSfxWfI4sdye3WIHk_0/edit?usp=sharing>

Here are the look-fors in the Danielson model:

<https://docs.google.com/document/d/1KGprDP6-7uNIh2mevLtvvoo1N2vrsapSCf9Y0LuZTes/edit?usp=sharing>.

Hope this helps. Feels very modest to me, but modest beats nuthin'

Doug

Hi Joel,

Thanks for these ideas. I'll be on the look out for presentations at TIES and elsewhere on this topic. Might be a great inaugural project for the MASA tech group as well.

The PD having a measurable impact on teaching practices certainly has traction here. It will be fun to see how this is done. It's a district-wide goal for us, not just in the tech dept.

Thanks again,

Doug

Hi Scott,

Yeah, figuring out how to measure the actual impact of PD on the classroom is our big challenge. Looking at student work is a great way to determine this.

Doug