Statewide Data Warehouse Project Goes Wrong

The state of New York has gone to a statewide data warehouse to track assessment and accountability data and reporting for K-12 students. The problem for many of us in the field is that the entire project was flawed from the beginning, creating chaos, poor documentation, rules on the fly, missed deadlines, late timelines, terrible reporting tools and reports, and incredibly poor communication. None of the processes or procedures were pilot tested, leading to the state deciding to go "live" with the data warehouse without knowing what worked and what didn't and not knowing whether applications being forced upon districts and Regional Information Centers were the appropriate tools to use to develop the reports used by districts to verify the accuracy of accountability/testing data. The State Education Department also never had a qualified project manager oversee or lead the project, which by all accounts is more complicated and involved than they ever could have imagined.

After two years of frustration preparing student management systems and district personnel for the collection and extraction of data necessary for accountability reporting, and forcing districts to use a reporting tool called nySTART that is incredibly flawed, inconsistent and poorly designed, the State Education Department has finally decided to discuss what can be done to fix the problem. What is clear is that a fair amount of the data in the warehouse could be wrong, particularly for larger, urban school districts. What's also clear is that programs and applications that are so poorly designed or don't integrate well with the data warehouse may need to be scrapped all together as they have lost credibility with the users. For example, it isn't unusal for a report to take 20 minutes to run and then have data that isn't accurate. Administrators with very busy schedules aren't likely to give the application too many chances when they don't trust or believe in the accuracy of the data.

I believe that data warehouse can and eventually will be beneficial. The intent is good as are the people working on the project. However, the key ingredients need to be put in place, such as the process has to be collaborative, transparent, involve feedback and communication at all levels with incredible support and resources. With so much riding on this project it will be interesting to see where things go over the next few months.

Learning the Basics from my kids

As I sat around the past week or so and watched my kids surf the net, IM friends, play endless songs on Guitar Hero, and even play the occasional game of ping pong, it's always nice to get the reminder of what works and why, in the field of ed technology. The games and tools that inspire and capture the imagination and interest of our children all have at their core something that is motivational. In many cases, intrinsically motivational. I'm reminded of my graduate school days at Syracuse University, learning about the design of instruction, and the concepts related to good design. One of the often overlooked areas of design and one of the more basic models is that of John Keller, and his ARCS Model. It's the simplicity and logic behind the model that make it hard to argue with and incredibly valuable.

Keller believed that good instruction should be motivational, and that the key concepts for his motivational theory involved gaining attention, making the learning relevant, building confidence in the learner/user, and finally, the satisfaction or reward the learner gets from the experience. Think about any of the tools that our children use, such as IM, or Guitar Hero, and you'll find all of the ARCS elements neatly folded into the applications. We don't have to make learning or the design of learning more difficult, we need to make it more relevant, practical, useful, and satisfying. Then, we're on the right track.