Twin Peaks — Measurements & Goals: Pros and Cons of Using High-Level Portfolio Milestone Timeliness Metrics with Linda Sullivan and Keith Dorricott
Podcasts Menu
EPISODE 14
How can you best measure whether vendors are meeting study milestone goals? That’s one of the important issues discussed in Keith Dorricott’s interview with Linda Sullivan, MBA, Executive Director of WCG’s Metric Champion Consortium (MCC). Dorricott is a Director at Dorricott Metrics & Process Improvement Ltd in the UK and facilitates many MCC metric development and community of practice groups. Timeliness metrics — measurements that assess whether study milestones are achieved by their target dates —- are valuable but don’t always tell the entire story, according to Sullivan and Dorricott, such as whether the milestone dates are realistic and how on-time measurements influence human behavior. Then, when you move from measuring a single study to cross-studies or portfolio level, aggregation of data can get even more complicated. There are many ways to “cut” the data, they note, depending on the performance questions you want answered. With timeliness, Dorricott says, it always comes back to expectations — and whether those expectations are reasonable. In this respect, measuring how long it takes to achieve milestones is important and should be reviewed to determine if timeliness expectations are achievable. In terms of rolled-up data, it’s easy to aggregate results across studies but it’s important to determine which studies or process steps are not meeting expectations and need attention. When you aggregate results from a large number of studies, it can be challenging to see when you have a few studies that are not performing well. For this reason, the MCC Vendor Oversight Metric Toolkit includes several portfolio-level milestone timeliness metrics that answer difference performance questions, such as what the overall performance of the portfolio is in terms of milestones met across active studies, performance over the last 3 months and performance reported by study phase.