I agree with your conclusion that there is a need to measure productivity, and it is this lack of an objective measure which makes the RTO decision hard to make.
The most common methodology I have seen at technology firms is some version of the Agile methodology which recommends dividing work into smaller tasks called stories (a repurposing of the small batch sizes idea from Lean ) and measuring the quantum of work by 'story points', indicating the effort required to complete the work by an individual. And then measuring the productivity of a developer or team by the number of story points completed over a quarter. This sizing of work is quite arbitrary and any obvious methods of sizing this like lines of code don't really work (although tech leaders asking for lines of code completed by each developer is not unheard of)
Since the task sizing is arbitrary there is no way to objectively measure productivity. Teams have an incentive (and the wiggle room) to inflate story points making it almost impossible to objectively measure productivity.
I think in the absence of any objective measures management has decided to anchor on the vibes of what productivity meant before the pandemic and wish to return to that. Teams like Cisco and some other firms that were fine for remote work before the pandemic are fine now too.
Story Points should be used to forecast capacity vs backlog only and not measure productivity. They are dimensionless. The estimate for a story is set purely by its relative size to other stories and therefore different teams can have widely different values (and that's ok). In such a system, if you don't try measure productivity, the sizings will naturally stablise around a set of relative values irrespective of any productivity growth. On the other hand, if you use story points to measure productivity, it will show an artificial rise as it's trivial to game, and your ability to forecast will be destroyed.
This all makes sense. One thing I was trying to parse was what you had implied by "dimensionless" -- i.e., they lack time units or that they are only comparative. Have seen some usage in using story point to estimate time to clear backlogs.
They should be purely comparative. A mistake most people make is to think in man-hours and then translate into Story Points. You can use them to estimate time by measuring 'velocity' which is the number of Story Points consumed per week (and then dividing backlog by velocity). So to work you need a good degree of stability to allow relative sizing over a body of stories and to allow velocity measurement over a period.
Thanks. I do think the small-batch breakdowns and story point measurements are good ideas with potential.
I find embracing some amount of subjectivity seems to be a sticking point. (Perhaps, subject for a future post, the trouble US manufacturing has had with Toyota Production system principles is probably do with adopting heavily subjective components. So objective ideas like kanban cards make it to whitecollar project management, but it is hard to translate autonomation or poka yoke.
More important than short term productivity measurement for RTO vs. not is the longer term consideration of how chemistry is built among high performing teams for the long haul. Evolution has clearly programmed for that to happen in person.
Fair point. (In academia too, the in-person effect is strongly felt in seminars and classroom teaching). Building team chemistry actually a strong point in support for RTOs and being in person. In fact, Jassy's 2023 announcement was very much on that principle.
One question is how frequently and how long do teams need to meet in-person to build performance through chemistry. It relies on subjective measures. I just think it may be possible with some hybrid model (i.e., people "work" 1 day away), but that hypothesis is being clearly stress-tested in real life now.
That chemistry in most societies would’ve been at home with your family, it’s not like hunters were out in the woods 40 hours a week with their buddies doing a 9-5 hunt, agrarian societies - at home working with family, anything pre-industrial would’ve had a majority time around working with family
Excellent piece about a prescient topic.
Thank you, Ross. Much appreciated.
I agree with your conclusion that there is a need to measure productivity, and it is this lack of an objective measure which makes the RTO decision hard to make.
The most common methodology I have seen at technology firms is some version of the Agile methodology which recommends dividing work into smaller tasks called stories (a repurposing of the small batch sizes idea from Lean ) and measuring the quantum of work by 'story points', indicating the effort required to complete the work by an individual. And then measuring the productivity of a developer or team by the number of story points completed over a quarter. This sizing of work is quite arbitrary and any obvious methods of sizing this like lines of code don't really work (although tech leaders asking for lines of code completed by each developer is not unheard of)
Since the task sizing is arbitrary there is no way to objectively measure productivity. Teams have an incentive (and the wiggle room) to inflate story points making it almost impossible to objectively measure productivity.
I think in the absence of any objective measures management has decided to anchor on the vibes of what productivity meant before the pandemic and wish to return to that. Teams like Cisco and some other firms that were fine for remote work before the pandemic are fine now too.
Story Points should be used to forecast capacity vs backlog only and not measure productivity. They are dimensionless. The estimate for a story is set purely by its relative size to other stories and therefore different teams can have widely different values (and that's ok). In such a system, if you don't try measure productivity, the sizings will naturally stablise around a set of relative values irrespective of any productivity growth. On the other hand, if you use story points to measure productivity, it will show an artificial rise as it's trivial to game, and your ability to forecast will be destroyed.
This all makes sense. One thing I was trying to parse was what you had implied by "dimensionless" -- i.e., they lack time units or that they are only comparative. Have seen some usage in using story point to estimate time to clear backlogs.
They should be purely comparative. A mistake most people make is to think in man-hours and then translate into Story Points. You can use them to estimate time by measuring 'velocity' which is the number of Story Points consumed per week (and then dividing backlog by velocity). So to work you need a good degree of stability to allow relative sizing over a body of stories and to allow velocity measurement over a period.
Thanks. I do think the small-batch breakdowns and story point measurements are good ideas with potential.
I find embracing some amount of subjectivity seems to be a sticking point. (Perhaps, subject for a future post, the trouble US manufacturing has had with Toyota Production system principles is probably do with adopting heavily subjective components. So objective ideas like kanban cards make it to whitecollar project management, but it is hard to translate autonomation or poka yoke.
More important than short term productivity measurement for RTO vs. not is the longer term consideration of how chemistry is built among high performing teams for the long haul. Evolution has clearly programmed for that to happen in person.
Fair point. (In academia too, the in-person effect is strongly felt in seminars and classroom teaching). Building team chemistry actually a strong point in support for RTOs and being in person. In fact, Jassy's 2023 announcement was very much on that principle.
One question is how frequently and how long do teams need to meet in-person to build performance through chemistry. It relies on subjective measures. I just think it may be possible with some hybrid model (i.e., people "work" 1 day away), but that hypothesis is being clearly stress-tested in real life now.
That chemistry in most societies would’ve been at home with your family, it’s not like hunters were out in the woods 40 hours a week with their buddies doing a 9-5 hunt, agrarian societies - at home working with family, anything pre-industrial would’ve had a majority time around working with family