Skip Ribbon Commands
Skip to main content

 Email  Share  Print

The Metrics System: Seven Smart Choices About Data

9/26/2016

Stephen G. Pelletier

computers

 

​With more technology comes more data. With more data comes the need for more technology to make sense of it all. Without experience, insight, and a firm fix on their missions, colleges and universities can get swamped and ultimately paralyzed by the flood of information. But more and more institutions are making smart choices about how and when to use data to improve campus efficiency and more effectively educate students.

 

Approximately a decade ago, Georgia State University (GSU) had a problem: Too few of its students were leaving with a credential—only about a third graduated within six years, and rates were even lower for underserved populations. To make matters worse, the campus lacked the tools to identify the factors that could move those indicators in the right direction.

GSU’s next move was to get serious about data, diving into information that could, for example, highlight areas where tutoring and other interventions could get a potential dropout back on track to graduation: The university analyzed 140,000 student records to pinpoint 800 factors that affect student success, and instituted a system that tracks those factors every night, triggering alerts for at-risk students who get help from an adviser within 48 hours.

Since then the strategy, along with other student success initiatives, has driven GSU’s six-year graduation rate up 22 percentage points, with the fastest increases among African American, Hispanic, and low-income students. The average time to degree has been reduced by almost a full semester, and the university has eliminated the graduation rate gap among students of different racial and economic backgrounds.

The data-driven approach helped GSU produce none of the most dramatic graduation rate increases in the nation, but the campus is no longer alone: Colleges and universities all over the country are learning to use data strategically, finding better ways to manipulate and analyze information across a spectrum of institutional activities ranging from supporting student success to improving a wide swath of business practices and bolstering overall institutional effectiveness.

It’s fair to say that in the race to improve data-driven student and institutional outcomes, some institutions are crawling while others are walking—and a few might be said to be running. Results from EDUCAUSE surveys in 2012 and 2015 about postsecondary data analytics showed that many institutions self-reported only modest progress over that time. One possible explanation, said Betsy Tippens Reinitz, director of the Enterprise IT program at EDUCAUSE, is that as institutions learn more about the world of big data, they also recognize how much more they need to do to fully tap data’s potential.

“I think where we are right now is that institutions are starting to really understand the value of data, analytics, and data-driven decision making,” Reinitz said. While she sees ample evidence that many college and university executives are convinced of data’s value, she finds that many institutions are still finding their way in learning how to use data effectively and strategically. To that end, for example, a recent paper from ACE, Evolving Higher Education Business Models: Leading with Data to Deliver Results, framed a college and university business model based on keeping financial data transparent, agreeing about metrics, and empowering front-line staff to make data-informed decisions that improve institutional practices.

Ensuring that their institutions are fully taking advantage of all that data, and using it to advance institutional priorities such as increasing the attainment rates of underrepresented students and meeting the needs of an increasingly diverse student population is a top priority of most college and university presidents, said Molly Corbett Broad, president of ACE. “Properly leveraging data to support institutional missions will be a game-changer for the future of higher education,” Broad said.

While building experience in using data, institutions have begun to accrue best practices and strategies that position them to leverage mission-critical information in support of institutional strategy and planning. In recent conversations, campus experts offered big-picture advice in seven broad areas:1

1. Cultivate an evidence-based culture.

Before they can make the most strategic use of data, some institutions may need to first develop a campus culture and decision-making practices predicated on the use of evidence and metrics. “I think in higher education we sometimes make decisions based on anecdote—the ‘we’ve always done it this way’ approach—and it can be hard to get out of that mindset,” Reinitz said. To rise above such barriers, she suggests, an institution may need to consciously develop a more evidence-based culture. Ironically, perhaps, some experts say that because many academics come from evidence-based disciplines, showing them how the institution is using data to drive decisions can help align campus support for new data-based practices.

In the future, some institutions may look a little like Indiana University–Purdue University Indianapolis (IUPUI), which has consciously nurtured “a strong culture of evidence-informed practice,” according to Kathy E. Johnson, IUPUI’s executive vice chancellor and chief academic officer. Noting that data at IUPUI is widely available to many different stakeholders, Johnson said that “using data to guide us has been a hallmark of how we strive to continuously improve.”

As another example, GSU has developed several highly successful data-driven student success initiatives. Essentially re-envisioning how it provides academic support, GSU recognized that different cultural norms were needed to inculcate the new practices and ensure their success. “We’re not talking here about just turning a switch on some new technology, but creating a change in culture on campus so that we behave and are organized in a different way than we were before the program launched,” said Tim Renick, vice president for enrollment management and student success and vice provost at GSU. Underscoring the vital role that top leadership can play in effecting cultural change, GSU President Mark P. Becker invested time in talking with campus staff at all levels about why new ways of working were necessary.

2. Clarify your goals.

A common pitfall in the use of data is allowing the promise of a particular new technology to drive institutional strategy. To avoid that misstep, colleges and universities need first to prioritize what it is that they want to do with data. “Agreeing on what your questions are is a really important part of the puzzle,” Reinitz said.

Aaron Walz, who directs the Business Intelligence Competency Center at Purdue University (IN), said that part of his shop’s mandate is to help coordinate how Purdue should focus its data and information priorities and activities. A salient consideration, he said, is, “What kinds of projects should we be doing, and how do they all fit together in helping us get toward the future state we’re trying to get to?”

At California State University, Fullerton, where data analytics have helped the institution increase graduation rates and narrow student achievement gaps, it is imperative to “make data actionable,” said José L. Cruz, who was until recently Fullerton’s provost and vice president for academic affairs. (He recently took office as president of Herbert H. Lehman College of The City University of New York in the Bronx.)

To help it assess its needs for business intelligence, the University of Iowa (UI) conducted in-depth interviews with 35 top executives. Campus leaders were asked to clarify their strategic and operational data needs and to discuss what analytics they wanted but didn’t have. The conversations helped frame “campus-wide communication and collaboration on strategic and operational data needs, policies, and procedures” and ultimately helped shape UI’s roadmap for business intelligence, said UI Database Administrator/Data Architect Brenda L. Ulin.

Another expert, John E. Sawyer, the associate provost for institutional research and effectiveness at the University of Delaware, advocates that colleges and universities develop and track data metrics linked directly to their strategic plans. Otherwise, he said, institutions can “wind up drinking from the firehose of our data, so to speak, and not really making a lot of sense of it.”

3. Build data communities.

On many campuses, control of data may have defaulted to those who collect it or have the tools to manipulate it. That’s not necessarily the most effective administrative model, and it might not get data into the hands of everyone on campus who can benefit from it, in formats like dashboards that enable them to draw insights they need to make decisions.

Addressing this issue, Purdue University has invested considerable energy in shaping a distributed network of campus data users. While the university’s business intelligence, institutional research, and teaching and learning technology groups collaborate closely to develop and execute university data priorities, Purdue is intentional about making data widely accessible to stakeholders across campus who get strong support in learning how to use that information to make decisions in their own work.

“Data really doesn’t work well when it’s sitting there in its own silo,” Reinitz said. “That may be even more important for analytics than for anything else. You don’t want the technical people driving analytics, at least in terms of what you are really using it for. That has to come from the business use case.” Making sure that business needs drives the technology and not the other way around requires governance structures that support and nurture that kind of cross-functionality, she said.

4. Hire the right people.

Many institutions avidly seek to recruit talent who can make sense of data and convert insights into actions that improve the institution. Finding and retaining the right expertise is challenging in today’s competitive market for IT professionals. But as hard as that may be, finding adequate technical skill often isn’t enough. IUPUI’s Johnson said leaders of campus data work need to have exceptionally strong communication skills. Ideally, she suggests, staff should have the ability to work with departments to understand their needs as well as the capacity to translate business case needs to IT.

Brent Drake, the chief data officer in Purdue’s Office of Institutional Research, Assessment, and Effectiveness, suggests that institutions do more to tap expertise in their institutional research (IR) offices. While those offices often focus on historic reporting and regulatory compliance, he said, IR staff have unique skill sets that could also be tapped to “take a deeper look at some of the efficiencies and effectiveness of the institution and how we best use our resources to build the best environment.”

5. Blend quantitative and qualitative.

One extreme of a focus on data is to place too much value on the numbers themselves. But algorithms can only tell you so much. Human analysis and contextualization of the numbers is vital. Noting that having data in hand sometimes creates a strong urge to act, John Carroll University (OH) Vice President for Enrollment and Institutional Analytics Brian G. Williams counsels institutions to instead allow time for analysts to develop deeper insights about what the data may mean. “The data is going to tell you a story, but the human context is going to help inform what you do,” he said. “It’s this blending of data from a quantitative as well as a qualitative perspective that interact to make good strategic decisions.”

6. Expect pushback.

Prodding institutions of higher education to develop a data-rich and evidence-based culture is bound to encounter some resistance. Some naysayers will argue that higher education is about people, not data. Staff used to traditional ways of working may have trouble adjusting to new directions.

Another potential hitch is a bit paradoxical: While we turn to data in search of answers, data often begets more questions. “I think sometimes people moving toward more use of data sense that this is going to be really efficient, and we’re going to have all this data mining tell us what to do. But that’s not what I’ve seen and experienced,” Williams said. “People say, look, we have all the right data and this is what it’s telling us, but then others will say, yes, but did you look at that or this. It opens curiosity in a wonderful way and creates a culture of questioning.” Williams notes that while that creative tension is valuable, even essential, it can nonetheless slow decision making.

7. Invest adequately.

While improved technology is bringing less expensive solutions online, robust data collecting and analysis requires a significant financial investment in people, expertise, and technology. “It’s easy to say yes, let’s make analytics a really important part of our institution, but you have to put your money where your mouth is and put some resources into that,” Reinitz said. In particular, institutions may have to pony up for competitive salaries for data experts. Ongoing support for data staff development is also crucial.

Increased use of data begets other concerns, of course, such as issues around data privacy and technology governance. But institutions that want to hone their use of data and use that information more strategically might start by making sure they have covered their bases in these seven critical areas.

José L. Cruz boils the challenges of making data operational and strategic down to this advice: “You have to have clear goals, you have to organize your human resources to drive those goals, you have to make sure that teams have actionable data, and you have to signal that it’s a top priority for the institution.” Other experts suggest that early “wins”—showing specifically how data can improve practices—can pave the way for further reforms. Champions who can advocate for the power of data are also invaluable.

Ultimately, the most important thing may be just to act. Don’t let data governance get bogged down in developing product specifications or data protocols or security layers, Williams counsels. “Start doing the work,” he said. “You will learn from it, gain momentum, and learn what next questions to ask.”

 

 Notes:

1 Colleges and universities today use data to improve practices across the enterprise, from learning to business practices. This article looks not at specific applications but rather at how institutions can make more strategic use of data writ large, and therefore uses terms like “data analytics” in their broadest contexts.

 
 
 
Stephen G. Pelletier is an independent writing and editing professional in the Washington, DC area.
 
 

Data-Driven Results

 

Data collection and analysis can dramatically improve a college or university’s processes, but student outcomes ultimately matter most. Among the highlights:

  • Indiana University–Purdue University Indianapolis (IUPUI) supports income-eligible students in Indiana’s 21st Century Scholars program with mentoring, workshops, and other assistance. “Through careful analysis of what works we’ve been able to support the continuous improvement of persistence,” said IUPUI’s Kathy E. Johnson. That analysis helped the program produce a 6 percent improvement in retention for this cohort in 2014 compared to 2013.
  • Piloting a dashboard that pulls student data from various databases and utilizing Education Advisory Board predictive analytics expertise, California State University, Fullerton (CSUF) has pushed its graduation rate up 10 percentage points to 62 percent and lowered its achievement gap, which had hovered near 12 percent, down to 9 percent.
  • By building a “finance data mart” and linking it to its business intelligence infrastructure, Purdue University (IN) eliminated the need to produce hundreds of operational reports. Among similar efficiencies, Purdue replaced a weekly admissions dashboard that was built manually in Excel, requiring 20 hours of staff time weekly, with an online automatic feed. Now, any dean or department head can track an incoming class the moment the data refreshes.
  • The University of Iowa incorporated a predictive model into dashboards it uses for enrollment management, providing modeling data that analysts believe provides a clearer snapshot of the potential size of the incoming class and better insights that help inform financial aid decisions. Insights from data about prospective and admitted students also help the institution save money by targeting mailings more precisely.

 

 
Subscribe to the Presidency 
2016 Fall Cover Image
Features
Columns