Cadmore media player playing video What Publishers Need to Know About Software Development
Video Player
Accessibility
Using forms mode, you can utilize keyboard shortcuts to make usability more efficient.
Space bar will play and pause the video. Pressing escape will close all displays and pressing it again will jump to the end of the player.
You can press A to open the Keyboard Shortcut Dialog which also provides an audio description of the player and all of its features.
Language
1x
Play Speed
Adjust Volume
Transcript
Search
Result Count
of
Click to jump to result
Search
Re-center
Segments
Result Count
of
Click to jump to result
Search
Name:
What Publishers Need to Know About Software Development
Description:
What Publishers Need to Know About Software Development
Thumbnail URL:
https://cadmoremediastorage.blob.core.windows.net/d34961dd-0260-4a11-b6fb-8f7e3a7097d7/thumbnails/d34961dd-0260-4a11-b6fb-8f7e3a7097d7.jpg
Duration:
T00H43M39S
Embed URL:
https://stream.cadmore.media/player/d34961dd-0260-4a11-b6fb-8f7e3a7097d7
Content URL:
https://cadmoreoriginalmedia.blob.core.windows.net/d34961dd-0260-4a11-b6fb-8f7e3a7097d7/Sequence 01.mp4?sv=2019-02-02&sr=c&sig=4RPQSdrdOlgvtn%2FFRQn%2F2DTZI7m2w4U%2Fjxlr0f8Z08c%3D&st=2025-01-22T11%3A42%3A46Z&se=2025-01-22T13%3A47%3A46Z&sp=r
Upload Date:
2020-11-18T00:00:00.0000000
Transcript:
Language: EN.
Segment:0 .
STUART LEITCH: All right, welcome, everybody. We're going to talk today about software development and some tidbits that you might found find relevant to you. Software development as a field is young, but it's also very complex. And while it's been extensively studied, academia has been less successful in really wrapping its arms around the field. And that's in part because software development is intrinsically less bounded than other engineering disciplines.
STUART LEITCH: It exists in an artificial, less constrained realm. And if you think about it, the medium of design and the medium of construction are actually the same. And that's not true of most engineering disciplines. But what most researchers do agree upon is that there's no process that consistently delivers reliable, dependable and affordable software systems. Now, you probably won't hear the consultants pitch that to you, because they're trying to pitch their latest framework.
STUART LEITCH: But it is the unfortunate reality of our field. And I've been doing this for most of my professional career and I've got some observations that I wanted to share with you. And these observations are related, but they're actually disparate topics. So it's really an assortment of insights that I thought might have some intersection with your various businesses.
STUART LEITCH: But the challenge with it being somewhat disparate is that retention in a context like this just period is very, very challenging. Like you have a presentation like this and then somebody does a survey a half an hour later and you actually only really remember about three things. It becomes more challenging when these disparate topics. And you may have heard the advice, tell them what you're going to tell them, tell them, and then tell them what you told them.
STUART LEITCH: Well, I'm going to do a variant of that today. But I'm going to do it slightly unconventional. And I'm going to break two rules of ballroom presentation in the process. Number one, I'm going to summarize for you everything I'm going to tell you as one enormous block of text that fills an entire slide. And even worse than that, I'm going to have Susan read it to you.
STUART LEITCH: And that's going to create some cognitive dissonance, because you can read faster than she can speak. But we're going to do a little experiment here, and I have a reason to think this might work for this kind of material but we'll find out. So we're going to gather some anecdotal evidence to see if this works. And the reason why I think it might work is there was a study done in Germany a couple of years ago where this strongly indicated that comprehension and retention are actually highest while simultaneously reading and listening.
STUART LEITCH: So this study was done in a controlled environment, and I haven't tried it in a presentation. But we are going to try it here. But the larger point here is that there's a lot of recommendations that are out there in terms of how you should develop software. But they're all contextual, and you have to be willing to experiment. You actually have to have the courage to deviate to try to really adapt to your organization.
STUART LEITCH: And this ties into the notion of software development as a less mature discipline. So unlike say mechanical engineering where a lot of the basics have long since been settled, software engineering being intrinsically less bounded, there's far less consensus about the methodology and in construction then in more mature disciplines. And in some respects, I actually liken software development to being more similar to the diet and weight loss industry in terms of the disconnect between academia and who's actually pushing the information.
STUART LEITCH: What's actually out there is primarily what sells conferences out, what sells books, what sells training, what sells consulting. So it's actually, you have to really be able to sift through that to get to the essence of what really matters. So at the end of this presentation, I'll ask for a show of hands, and we'll kind of see if this experiment kind of actually aided.
STUART LEITCH: And I actually just wanted to be a genuine experiment. You need to be honest with yourself about what works. So the next slide is basically going to be this wall of text. It's not all going to make sense at first. But we'll basically pick it apart through the next half an hour, and then we'll see if we actually undistorted at the end of the day. So I'm going to have Susan read this out to you.
SPEAKER 2: "Software development is an industry with rampant cost and schedule overruns. Its high complexity means we need to explicitly manage what we don't know, understand that scope expands fractally with detail and that requirements volatility must be managed. Estimates are probabilistic, but actuals are right skewed. So this needs to be factored in. Adding people to a late project makes it later because of the communication between nodes.
SPEAKER 2: How you organize will be driven by trade offs between specialization, communication overhead, and cycle time. Colocation versus distribution and culture matter a lot Agile is largely a capitulation to uncertainty in the face of complexity but is pragmatic if you don't buy into the hype. Schedule compression is a killer, an iteratively MVP if you can.
SPEAKER 2: If you want to move fast, you must be willing to break things or spend a lot of money. Be pragmatic about the long tail of edge case defects, and don't sweat them. Try to think in terms of long term cost of ownership and hew to the happy path, deviating with customization for decisive value drivers. Finally, effective engagement of knowledge workers is what outweighs all else."
STUART LEITCH: Thank you. So that's pretty much it. And then we're just going to walk through this. So I'm going to have Susan-- before each slide, she'll just read the sentence that's relevant, and then I'll basically give some context about that.
SPEAKER 2: "Software development is an industry with rampant cost and schedule overruns."
STUART LEITCH: So, these are results from a large study that was done in collaboration between Oxford where my boss Tim Barton, wherever he is-- great boss by the way, great boss-- and McKinsey. My brother's a partner there. So these two institutions got together. And they studied around 3,000 software projects with a median value of about $2 million. And what it shows is fairly dismal.
STUART LEITCH: Two out of three software projects in this study face cost overruns. And the average cost overrun was 78%. We also had four out of five experience a schedule overrun. Now, the overrun for those that did overrun wasn't this bad. That was 35%. But regardless, this is a really messy industry. And it's not an industry with high predictability for large projects.
SPEAKER 2: "Its high complexity means we need to explicitly manage what we don't know."
STUART LEITCH: So, the nature of software development in a complex ecosystem is that as human beings. We're not actually able to fully reason in advance about the full implications of system changes. There's certainly software engineers that pretend that they do and have the confidence. But the reality is that we know empirically that we're not particularly good at that. And I think this is in part that we're just cognitively not well adapted to this level of complexity.
STUART LEITCH: And how it actually works is that our knowledge expands as we get in contact with the work. But we have to kind of think about that there's that famous Donald Rumsfeld quote about the known knowns, the known unknowns and then the unknown unknowns. It's those latter two that you really need to be cognizant of with software development and particularly the unknown unknowns.
STUART LEITCH: There's a lot of traps here way you think you actually know something. You've gone into this, and you kind of understand the details, and you've done something very similar to this in the past. And you often run into traps where you get down into the details and there's some subtle and patents mismatch between systems and you spend an extraordinary amount of time just walking around some of those nuances.
STUART LEITCH: And we see how humbling it is. Even on small tasks, you can be easily an order of magnitude of in terms of your estimates. And we do actually have a model for understanding this. It's called the Cone of Uncertainty. Who's heard of the Cone of Uncertainty? About half the room so time going left to right, it's basically your uncertainty around budget and schedule narrowing as you get further down to the project.
STUART LEITCH: And it's probably true in many industries but we like to cite a lot in software that you will actually know what the project is going to cost and how long it's going to take when you're done. And that is a bit of a problem. But related to this, what's interesting is that if you look at books around software estimation-- and they are kind of interesting books to read.
STUART LEITCH: There's no magic formulas in there, it's just basically how do you bring more and more attention to the estimation process. But one of the basic practices they talk about is bracketing with high and low estimates. And in part, this is designed such that you're actively actually thinking about the uncertainty. Well, what's interesting is that academia has been able to then go and study, though, that bracketing relative to actuals.
STUART LEITCH: And at least the studies that I looked at suggested that the bracketing actually wasn't wide enough, that the brackets were too narrow. So that's saying that even when we're explicitly trying to account for uncertainty, we have a hard time fully accepting the reality of it. And so the takeaway here is that we really need to explicitly manage what we don't know.
SPEAKER 2: "Understand that scope expands fractally with detail."
STUART LEITCH: So determining the length of the country's coastline sounds pretty straightforward, but it's not. The answer actually depends on the length of the rule that you use for the measurements. The shorter the ruler, the more bays and inlets you capture than if you had a larger ruler. And so as you essentially measure in more and more detail, the length of the coastline actually expands. And there truly is no answer to what's the length of the coastline.
STUART LEITCH: It actually depends on the measurement. And this applies equally to software requirements. As you get further and further down into the detail, it expands out fractally. And you just want to be really aware of that issue going in. And if you're going to be really prescriptive at a super detail level, you need to make sure that you've really allowed for that upfront or that you're not you're killing off scope somewhere else.
SPEAKER 2: "Requirements volatility must be managed."
STUART LEITCH: So as you get into a project, you should expect that your understanding of it is going to evolve. And that's likely going to drive changing requirements. That's actually a good thing. But it can also be a really, really toxic thing. And the challenge is that when you change requirements, it often introduces rework, it creating efficient, you're having to kind of go through a whole lot of communication to get people realigned around that.
STUART LEITCH: And you'll often have to go and cut scope elsewhere to have the project remain right sized. And depending on who your stakeholders are kind of in the middle of the process, you also need to really watch the tendency to try and perfect everything. As you get greater knowledge, you can always rework things, you can always come up with a better solution. And it's very easy for changing requirements to get so out of hand that they actually threaten the entire project.
STUART LEITCH: So the message here is that you want to be really disciplined around what you let change in a software project.
SPEAKER 2: "Estimates are probabilistic, but actuals are right skewed. So this needs to be factored in."
STUART LEITCH: So this is a slide from a few years back. This is on us household income distribution. And the first thing you can notice is that it's not a normal distribution, it's actually right skewed. And that kind of make sense. If you think about us as a wage earner, well, you're really earning zero dollars or above. So it's low bounded, but particularly when you think about stock traders or elite athletes, there's really there's no real upper bound here.
STUART LEITCH: And I mentioned yesterday-- well, actually yes, it was yesterday-- lies, damn lies, and statistics. Now this wasn't too impugn that statistics aren't a developed branch of mathematics. It was really to say that without careful treatment, it's very easy to draw the wrong conclusions. And if we were to use a single statistic to describe income on this curve, what would we use?
STUART LEITCH: If we thought about a random household, well, the most likely income they going to have is going to be between $15,000 and $20,000, because that's the mode. If we were to split the population in half, we'd have $50,000, that's the median. But then if we were to look at the average, well, the average is $72,000. And there's really not a right answer.
STUART LEITCH: In many respects, if you're actually working with these questions, it probably depends on what political party affiliation you have and what the agenda you're trying to drive is. But the point here is just basically thinking about different ways to describe these and the challenges that most people conceptually deal with averages, but their actual lived experience is with the thing that they most frequently experiencing, which is much more related to the mode and the median.
STUART LEITCH: And this ties into how we estimate. Because human beings tend to estimate based on their accumulation of direct experience. And so estimates actually tend to cluster around the median and towards the mode rather than the actual average. And if you ask your accountant what actually matters, well, he'll tell you it's the average. He doesn't care about the rest of this stuff. But when you've got a software team that's essentially estimating around, say, the median, you actually need to be able to account for that.
STUART LEITCH: And one of the things we've done at Silverchair is we have all of our internal data about our estimations and our actuals. And so we've actually been able to look at essentially the sum of the actuals relative to the sum of the estimates. And we do find that there's about a 30% delta there. And this actually happens for us to correspond approximately to the difference between the median and the average.
STUART LEITCH: And so the devil is in the detail with software development. It's very common for outliers to really drag the average to the right. And the purpose of an estimate is basically you're not looking for precision there. We know through the Cone of Uncertainty that if you really want precision, you've got to actually fully complete the work or accuracy I should say.
STUART LEITCH: The point is to basically get enough of a handle on the overall project that in aggregate it can be controllable. What we do is we periodically check in and do evidenced by estimation where we're looking at this and saying, OK, how far are we actually falling from our original estimates in terms of actuals? And then we just have simulators just make that adjustment. We don't make a big deal about it.
STUART LEITCH: We're just trying to recalibrate. So we're generally in the right ballpark.
SPEAKER 2: "Adding people to a late project makes it later because of the communication between nodes.
STUART LEITCH: So who has heard of this book? OK, quite a few people. Who's read the book? OK, a few people. Well this is a classic this is Fred Brooks from 1975, the year I was born. And it is filled with nuggets. So this was about Fred's time at IBM and got a bunch of timeless wisdom in it. But one of the headlining sites was that adding people to a late project makes it late.
STUART LEITCH: And the reason is primarily driven by communication overhead, which explodes exponentially as you add more people into a project. And he goes on to say that complex programming projects cannot be perfectly partitioned into discrete tasks without complex interrelationships. Now, if you listen to the folks with all the buzz around microservices, you would think that they've actually solved that problem.
STUART LEITCH: But if you speak to anybody that's actually done this at scale, they'll disabuse you of that notion. Microservices are helpful, but they're not a silver bullet. And there's kind of a consistent theme in software that there really are no silver bullets. And actually, that was Fred Brooks's next book, No Silver Bullet. So, go on to the next slide.
SPEAKER 2: How you organize will be driven by trade offs between specialization, communication overhead, and cycle time.
STUART LEITCH: So we just talked about communication overhead. But specialization it's really foundational to the productivity gains of the modern economy. Specialization by and large is a really good thing. But the challenges is as people become increasingly specialized, you then need to add more people in, which then places a lot of pressure on your communication overhead. And so you then essentially stop petitioning your work to create smaller teams.
STUART LEITCH: And that's where kind of things like microservices kick in. But the challenge is that we're in an increasingly rapid change environment within our businesses overall. And we see these cross cutting influences come in that actually require a lot of things to be changed across teams across services. And that's where you actually have to really think about cycle time, because microservices is definitely all the rage right now.
STUART LEITCH: But if you actually have to go and refactor across a whole bunch of microservices, you'll be asking yourself, how did the cycle time get so long? And that's actually the dance you have to do between these three trade off to really think about how change is likely to flow through your business and in what cycle time do you need to respond to that. And it was interesting what John Shaw said yesterday in the technology roundtable.
STUART LEITCH: Is John still here? No. But he was talking about how they are now requiring technologists to be able to interact with their customers. So they're narrowing down the labor pool that they can select from. But I suspect that it's really driving down their communication nodes and actually speeding up their cycle time.
STUART LEITCH: So that's a typical kind of trade that you have to think about.
SPEAKER 2: "Colocation versus distribution and culture matter a lot."
STUART LEITCH: So 200 years ago, if we all wanted to be in a room like this together, we'd basically have to get on our horses and kind of all come together. We've now got more modern modes of transportation, but we've also got fancy electronic communication where we could actually all be doing this virtually. And there's a lot of cost to get everybody in this room. So the question is, well, why didn't we just do this virtually?
STUART LEITCH: Well, that's a bit of a rhetorical question. There's actually a lot of intangibles here; number one, attention, like your actual engagement is a lot higher, there's the relationships that are being built, there's the relatedness, there's the conversations that are happening outside this room, not to mention the booze ups afterwards, which people rave about. The same dynamics are present within teams.
STUART LEITCH: In complex knowledge work, you're wanting to harness both the individual intelligence, but you're also wanting to harvest the collective intelligence. And that collective intelligence is really driven by the relatedness and the connectedness of your people. And so it's certainly possible to do this with remote teams.
STUART LEITCH: But like an event like this, it's a lot easier to do in person, so you just want to be really thoughtful of those dynamics. And related to this is the notion of power distance, or strength of social hierarchy. This is really about, does a person defer to authority, or do they challenge and upgrade perspectives coming from above? And as businesses are dealing with increasing change and complexity, traditional command and control management structures are really breaking down.
STUART LEITCH: And you want software developers that who are fully engaged in the process and not just order taking. And at Silverchair, we really think about this in terms of our hiring process. It's something that we explicitly screen for. We want people that can actually just push back and that are actually willing to do that, because it takes something to do that. Our labor pool is US-based, so it's actually a lot easier for us, because in terms of global averages, Americans tend to just-- we're a lot more willing to speak truth to power.
STUART LEITCH: Those of you that have offshore in India or other locations, you'll probably have the experience where they can go very fast in a particular direction. But if you're not really clear in the right direction, they're not going to tell you. And so this is just something to think about that it's a trap if you're not actively managing it when offshoring, because you want people that will challenge the plan when they say misalignment with the overall intent.
SPEAKER 2: "Agile is largely a capitulation to uncertainty in the face of complexity but is pragmatic if you don't buy into the hype."
STUART LEITCH: So this is basically a chart. And I shared this a couple of days ago with a few folks. There are a few overlapping slides here. But software development is basically about 50 years old. We say agile methodologies pop up in 1995 through 2000. Agile really only became a thing in 2001 with the Agile Manifesto. But we often hear about agile in contrast to waterfall. Well, that's a bit of a misnomer.
STUART LEITCH: It's that the methodologies from the 80s onwards actually had spiral or iterative dynamics within them. It was more that there were just so heavy weight. And there was a real breakdown in terms of the long range planning that was involved to actually produce predictable results. And agile was largely a rebellion against that. And what it did was explicitly optimized for change. It really shifted the posture and essentially said, OK, we're just going to operate in very short iterations, and we're actually not going to-- at least in terms of something like Scrum-- we're not explicitly going to even try and plan beyond a sprint.
STUART LEITCH: And that worked very well for small consulting teams and product teams. But it didn't work well at scale. And so we've seen an evolution of methodologies here that have added in more and more weight. And there's certainly been a lot of the weight from prior frameworks that slipped in here. And each framework will be sold to you as the next best thing since sliced bread.
STUART LEITCH: And there's typically a lot of fear around this. You'll see articles, are you doing agile right? There is no silver bullet. And these are really just guides that you need to select from and think contextually about what fits for you. This is the most popular framework for larger organizations, this is scaled agile. And this is starting to look pretty complex.
STUART LEITCH: How we at Silverchair look at this is we look at each of the prescriptions here and we ask ourselves, what's the intent of that prescription, how does that fit in our particular context? Because software development environments are complex, it's not one size fits all. And then we ask ourselves, does this prescription add decisive value for us? And ultimately, we use the adage, as much as necessary, as little as possible, because you have to keep pruning your process or you will calcify as a business.
STUART LEITCH: And these are the values that were the original Agile Manifesto. There's a little bit more to it. But you ask anybody that was there, and this is basically what actually mattered. And you want to keep these in mind if you're doing software development, you've got agile consultants in, because what they tend to sell is the processes, the tools, and the planning devices, because that's part of what they can sell.
STUART LEITCH: But this is actually really the essence of agile. And another way of looking at this is to think of agile really as a tight feedback loop. All of these processes are trying to facilitate throughout the organization people just being in a learning organization relative to the work that they're doing.
SPEAKER 2: "Schedule compression is a killer and iteratively MVP, if you can."
STUART LEITCH: So we talked earlier about the communication overhead that occurs and when you are trying to think about, well, how aggressive should I be with my project schedule here? Theory would tell us that if we get too aggressive with it, the overall relative effort that we incur is going to kind of hockey stick up there. And likewise, we also have this notion of Parkinson's law. Who knows Parkinson's law? Well, basically, work expands to fill the time allotted.
STUART LEITCH: And there's been some studies done on this. And this is a meta study that is essentially looking at these. And what it's showing is that schedule compression across these studies was absolutely found to be true. You can really compress schedule, but you're going to have to do it by throwing a lot of extra dollars in there-- inconclusive about Parkinson's law.
STUART LEITCH: MVP, MVP has now become a bit of a dirty word. It kind of is icky because sometimes people call something an MVP where it actually kind of really isn't. But if you've got the architecture and the business context that will support iterative development, it's generally a better way to go.
SPEAKER 2: "If you want to move fast, you must be willing to break things or spend a lot of money."
STUART LEITCH: So we typically want both speed and safety. But getting more of one typically comes at the expense of the other. Building in safety processes tends to slow things down. But it does reduce risk. And we need to be continually challenging the assumption that we've actually got the risk right and really understand what our risk tolerance is for the particular aspect of the business that's being impacted by the software development that we're working on.
STUART LEITCH: Now Formula 1 is one of those interesting kind of exceptions here where-- this is a crash obviously. But in general, relative to how fast they go, I'm actually surprised that there aren't more fatalities. It actually is relatively safe. Well, I actually did check with this photo, and they all walked away. They were actually unhurt, no hospitalization, nothing.
STUART LEITCH: That's pretty crazy. But it also costs. The cheapest team in Formula 1 is spending about $100 million a year. To be competitive, you apparently need to spend about 250 million a year. So there are real trade offs here. And most people are going to essentially find some balance between speed and safety.
STUART LEITCH: Now I'm going to kind of tell a personal anecdote here about my run in with software speed and safety. So this is an electric unicycle. And these are kind of marvelous things. There are an evolution of the Segway but just with one wheel. And it's like a briefcase with pedals on the side and a wheel in it. And you basically stand on it. And you have to balance it side to side, like a bicycle.
STUART LEITCH: And it sounds really hard, but it's actually just like riding a bicycle. But you then have to let go of your balance forward to back. And you basically fall forward. It detects that and it applies torque and gets underneath you. And then you fall back to slow down, remarkable thing. And one of my work colleagues got one of these. And he was zipping around on it, oh, it's so awesome, all this other stuff.
STUART LEITCH: And I was just like, whatever. Anyway, he has this accident on it, and he breaks his arm. And I'm like, bet you're not going to be doing that anymore. And he's like, oh no, I was really serious I'm just telling you how awesome this thing is. It's like a magic carpet. I mean, I go everywhere in Charlottesville on it. And so I learned more about it. And I learned that he actually had you know an older model that wasn't that sophisticated and actually used within it technology out of electric bicycle.
STUART LEITCH: And one of the things in that technology is that it's got overvoltage protection, just to really protect the longevity of your bike so that you don't wear out the components too soon. But the challenge with these wheels is that when you're riding along, you're typically going really fast. And you might hit like a rock on the road. And so that immediately pitches you forward.
STUART LEITCH: So the wheel has to surge with power to basically get underneath you. But what would happen with these bicycle components was it would detect the overvoltage and it would shut off the power. And it's even worse than it sounds, because the motors no longer actually a motor, it's a generator. But it's got no way to put the energy.
STUART LEITCH: So basically, the electricity can't go any way, so the wheel just basically immediately locks up. And it's got a fly swatter effect. And so it is actually pretty dangerous, but I was intrigued by this. And I went out, and I got myself a wheel. I got a current generation one, I got one that was really pretty safe. And I ended up putting over 1,000 miles on it zipping all around Charlottesville.
STUART LEITCH: And I actually stopped using my car to such an extent that I had to schedule it in my calendar to go and stop the thing so the battery wouldn't run down. Ultimately sold the car. And when I sold the car, I thought well I just need to get the best in the world at this, I have to get the best. And so it then became a whole mission to figure out, well, what was the best?
STUART LEITCH: And this is where I had this trade off to make, that there were two basic brands. They were both Chinese brands and this is in part because they just kind of relentlessly rip off the intellectual property of the US companies. So it's now just these Chinese companies. But you've got these brands that are known for different things. And King Song is essentially known for being much more conservative, and they're really safety first.
STUART LEITCH: And Gotway, on the other hand, they're just like sports cars, they push the envelope. And I was looking at this and trying to understand the differences. And one of the things-- they were both priced about the same, but Gotway was introducing a higher voltage. And higher voltage means more energy at the same current so you can squeeze more out of your wheel.
STUART LEITCH: And the thing that really jumped out to me, this one goes like 20 something miles an hour, this goes 30 miles an hour. And yet, Gotway actually had a reputation for being a little less conservative on the safety side. But I ended up deciding that I was actually willing to run that risk. And so I kind of signed up.
STUART LEITCH: They were releasing both these models. It just so happened to be that I was quite lucky. I missed out on the first shipment of products coming over from China, and I got a second batch. But while the second batch was on the way, all the people trying out the first batch, well, I started to have the fly swatter thing happen to them. And if you kind of Google the images around that time, it's pretty nasty, people like with teeth missing and all.
STUART LEITCH: But anyway, it turns out this was a firmware issue. They'd done all the prototyping, and it really worked, and they were just kind of fine tuning this stuff. And they just made some mistakes in the software. And this happens. They were rushing to market. And as a result of that, they just intercepted mine as it came out of customs in Los Angeles, they flashed the firmware.
STUART LEITCH: I've had it for three years, I love the thing. I've ridden over 3,000 miles on it now. But this is a really practical example. It's not just hypothetical that you actually have to really think about the speed versus safety dimensions. And a lot of you are running a lot of money over various platforms, and you really need to think about which dimensions of that are more susceptible to-- what's the impact of business disruption if aspects of that goes down?
SPEAKER 2: "Be pragmatic about the long tail of edge case defects, and don't sweat them." So any sufficiently complex software system has a very long tail of defects that are not economically viable to fix. We often don't talk about that, but it's just real. And closing issues as won't fix is common practice for all major software vendors. And I'm just going to give you two examples of this. So this is essentially, we're looking at Firefox here.
SPEAKER 2: Firefox is a great browser, very stable. Probably half the people in this room use it. The reason I picked Firefox is that their bug databases public, so we can essentially look at that. This is filtering down and saying, OK, this is just looking at defects for Firefox saying in the last 30 days, there were 119 defects closed as won't fix, that's how it works. This is Atlassian.
SPEAKER 2: We use the Jira and Confluence products. Specifically looking at Jira, in the last 30 days, over 200 bugs closed just won't fix. The point here is we all want quality. But we need to be really thoughtful about what we're pursuing in terms of quality, because there's definitely a trade off. And if you're sufficiently close to the product and you are looking for perfection, you're going to be disappointed, because you actually can't get all the long tail of defects out there.
SPEAKER 2: So you want to take a really blended approach. And you want to be really cautious of the OCD types that are involved in the project, because they can really wrap things around the axle. You're thinking about, what are the issues that actually really drive value versus the things that just drive perfection? "Try to think in terms of long term cost of ownership and hew to the happy path, deviating with customization for decisive value drivers."
STUART LEITCH: So we've all seen a slide like this. This is the concept of an iceberg. And the challenges that we know that there's a lot of stuff under the iceberg. We actually know that's where the bulk of its mass is, but we only say the thing at the top. And this is something to really think about when we customize software. And generic advice that I give to people is to think about what the happy path of that software is.
STUART LEITCH: How does this software actually want to work? And really try and map that onto your business needs and say, can we actually-- you don't want to just take some piece of software and radically adapt it so that it fits your world. You actually need to be willing to adapt to it as well, because the further you get off that happy path, you're going to start to see gaps open up in the user experience.
STUART LEITCH: You're going to see it become increasingly clunky. And you're going to spend a lot of money doing the configuration and the customization to get it there. And you're going to find more and more things break in the future. And so there's a real cost to customization that people often don't really own. And the point is customization should really be focused for the things that add decisive business value.
SPEAKER 2: "Finally, effective engagement of knowledge workers is what outweighs all else."
STUART LEITCH: So if agile is about decentralizing decision making and creating tight feedback loops, it's decisive that hearts and minds of those people who are actually in that process are aligned. And you want to get your workplace beyond a transactional environment when people have real heart in the game and experience enough psychological safety that allows them to show up with commitment and actively cause effectiveness in the feedback loops around them.
SPEAKER 3: "Software development is an industry with rampant cost and schedule overruns. Its high complexity means we need to explicitly manage what we don't know, understand that scope expands fractally with detail and that requirements volatility must be managed. Estimates are probabilistic, but actuals are right skewed, so this needs to be factored in. Adding people to a late project makes it later because of the communication between nodes.
SPEAKER 3: How you organize will be driven by trade offs between specialization, communication overhead, and cycle time. Colocation versus distribution and culture matter a lot. Agile is light largely a capitulation to uncertainty in the face of complexity but is pragmatic if you don't buy into the hype. Schedule compression is a killer and iteratively MVP, if you can.
SPEAKER 3: If you want to move fast, you must be willing to break things or spend a lot of money. Be pragmatic about the long tail of edge case defects, and don't sweat them. Try to think in terms of long term cost of ownership and hue to the happy path, deviating with customization for decisive value drivers. Finally, effective engagement of knowledge workers is what outweighs all else."
STUART LEITCH: Wow, I was almost overcome with cognitive dissonance. It is challenging, isn't it? Having somebody raid and then you're kind of reading ahead and you have to keep aligning with that. Well, I wanted to just get some feedback as to the pain of that, was that outweighed by the effectiveness? So I'd like to see a show of hands-- and just be honest-- do you think that this helped with retention of information and comprehension?
STUART LEITCH: OK, yeah, about 3/4 of the room. OK, well, fascinating. Well, I'll end this by saying I've shared you know some context that I've experienced through my career. But I actually need to apply to what I've said to you to my critique of almost everything in this industry. And that is that no rules are universal, maybe this one's universal.
STUART LEITCH: But no rules are universal, and all rules need context. And you really need to be continuing to try on what's out there and discard things that are no longer serving you. So thank you very much. I think we're pretty much at time
SPEAKER 3: I think we can take one question, one question. So we have time for one question. Anyone out there have a question for Stuart?
STUART LEITCH: I was sure somebody was going to ask me a unicycle question
AUDIENCE: Do you have one of those or not?
STUART LEITCH: I do. I actually went got a full face helmet. I actually had this conversation about was I willing to live with a busted up face. And I decided I'd just wear a full face helmet.
TIM BARTON: So I'm Tim Barton, I'm Stuart's excellent boss. sure I think people in the room, there may be people from large companies that do software developing themselves and then other models that you outsource. Have you got anything to say that might be useful to people in the room who are in the outsourcing side of side of things in terms of when they're not doing the software development themselves, but they're in a relationship with an organization that's doing software development for them, is there anything you'd say that would help them think about that?
STUART LEITCH: Yeah, I think software development is rarely a set it and forget it exercise. Even if you do big requirements planning up front, one of the things that we kind of see with the evolution of agile is that stuff never really worked all that well to begin with. And so you are going to have changing requirements. You are going to hit up against unknowns that are going to derail your project. And you're going to have to basically collectively get in there and wrestle this thing back onto the tracks.
STUART LEITCH: And so you want to be thinking about who are you actually going to be doing that with. Are you going to be doing that with a partner, or are you just going to be doing that in a traditional vendor relationship? Because it makes a remarkable difference what the relationship actually is.
SPEAKER 4: Thank you, Stuart and thank you, Susan.
Segment:0 .
STUART LEITCH: All right, welcome, everybody. We're going to talk today about software development and some tidbits that you might found find relevant to you. Software development as a field is young, but it's also very complex. And while it's been extensively studied, academia has been less successful in really wrapping its arms around the field. And that's in part because software development is intrinsically less bounded than other engineering disciplines.
STUART LEITCH: It exists in an artificial, less constrained realm. And if you think about it, the medium of design and the medium of construction are actually the same. And that's not true of most engineering disciplines. But what most researchers do agree upon is that there's no process that consistently delivers reliable, dependable and affordable software systems. Now, you probably won't hear the consultants pitch that to you, because they're trying to pitch their latest framework.
STUART LEITCH: But it is the unfortunate reality of our field. And I've been doing this for most of my professional career and I've got some observations that I wanted to share with you. And these observations are related, but they're actually disparate topics. So it's really an assortment of insights that I thought might have some intersection with your various businesses.
STUART LEITCH: But the challenge with it being somewhat disparate is that retention in a context like this just period is very, very challenging. Like you have a presentation like this and then somebody does a survey a half an hour later and you actually only really remember about three things. It becomes more challenging when these disparate topics. And you may have heard the advice, tell them what you're going to tell them, tell them, and then tell them what you told them.
STUART LEITCH: Well, I'm going to do a variant of that today. But I'm going to do it slightly unconventional. And I'm going to break two rules of ballroom presentation in the process. Number one, I'm going to summarize for you everything I'm going to tell you as one enormous block of text that fills an entire slide. And even worse than that, I'm going to have Susan read it to you.
STUART LEITCH: And that's going to create some cognitive dissonance, because you can read faster than she can speak. But we're going to do a little experiment here, and I have a reason to think this might work for this kind of material but we'll find out. So we're going to gather some anecdotal evidence to see if this works. And the reason why I think it might work is there was a study done in Germany a couple of years ago where this strongly indicated that comprehension and retention are actually highest while simultaneously reading and listening.
STUART LEITCH: So this study was done in a controlled environment, and I haven't tried it in a presentation. But we are going to try it here. But the larger point here is that there's a lot of recommendations that are out there in terms of how you should develop software. But they're all contextual, and you have to be willing to experiment. You actually have to have the courage to deviate to try to really adapt to your organization.
STUART LEITCH: And this ties into the notion of software development as a less mature discipline. So unlike say mechanical engineering where a lot of the basics have long since been settled, software engineering being intrinsically less bounded, there's far less consensus about the methodology and in construction then in more mature disciplines. And in some respects, I actually liken software development to being more similar to the diet and weight loss industry in terms of the disconnect between academia and who's actually pushing the information.
STUART LEITCH: What's actually out there is primarily what sells conferences out, what sells books, what sells training, what sells consulting. So it's actually, you have to really be able to sift through that to get to the essence of what really matters. So at the end of this presentation, I'll ask for a show of hands, and we'll kind of see if this experiment kind of actually aided.
STUART LEITCH: And I actually just wanted to be a genuine experiment. You need to be honest with yourself about what works. So the next slide is basically going to be this wall of text. It's not all going to make sense at first. But we'll basically pick it apart through the next half an hour, and then we'll see if we actually undistorted at the end of the day. So I'm going to have Susan read this out to you.
SPEAKER 2: "Software development is an industry with rampant cost and schedule overruns. Its high complexity means we need to explicitly manage what we don't know, understand that scope expands fractally with detail and that requirements volatility must be managed. Estimates are probabilistic, but actuals are right skewed. So this needs to be factored in. Adding people to a late project makes it later because of the communication between nodes.
SPEAKER 2: How you organize will be driven by trade offs between specialization, communication overhead, and cycle time. Colocation versus distribution and culture matter a lot Agile is largely a capitulation to uncertainty in the face of complexity but is pragmatic if you don't buy into the hype. Schedule compression is a killer, an iteratively MVP if you can.
SPEAKER 2: If you want to move fast, you must be willing to break things or spend a lot of money. Be pragmatic about the long tail of edge case defects, and don't sweat them. Try to think in terms of long term cost of ownership and hew to the happy path, deviating with customization for decisive value drivers. Finally, effective engagement of knowledge workers is what outweighs all else."
STUART LEITCH: Thank you. So that's pretty much it. And then we're just going to walk through this. So I'm going to have Susan-- before each slide, she'll just read the sentence that's relevant, and then I'll basically give some context about that.
SPEAKER 2: "Software development is an industry with rampant cost and schedule overruns."
STUART LEITCH: So, these are results from a large study that was done in collaboration between Oxford where my boss Tim Barton, wherever he is-- great boss by the way, great boss-- and McKinsey. My brother's a partner there. So these two institutions got together. And they studied around 3,000 software projects with a median value of about $2 million. And what it shows is fairly dismal.
STUART LEITCH: Two out of three software projects in this study face cost overruns. And the average cost overrun was 78%. We also had four out of five experience a schedule overrun. Now, the overrun for those that did overrun wasn't this bad. That was 35%. But regardless, this is a really messy industry. And it's not an industry with high predictability for large projects.
SPEAKER 2: "Its high complexity means we need to explicitly manage what we don't know."
STUART LEITCH: So, the nature of software development in a complex ecosystem is that as human beings. We're not actually able to fully reason in advance about the full implications of system changes. There's certainly software engineers that pretend that they do and have the confidence. But the reality is that we know empirically that we're not particularly good at that. And I think this is in part that we're just cognitively not well adapted to this level of complexity.
STUART LEITCH: And how it actually works is that our knowledge expands as we get in contact with the work. But we have to kind of think about that there's that famous Donald Rumsfeld quote about the known knowns, the known unknowns and then the unknown unknowns. It's those latter two that you really need to be cognizant of with software development and particularly the unknown unknowns.
STUART LEITCH: There's a lot of traps here way you think you actually know something. You've gone into this, and you kind of understand the details, and you've done something very similar to this in the past. And you often run into traps where you get down into the details and there's some subtle and patents mismatch between systems and you spend an extraordinary amount of time just walking around some of those nuances.
STUART LEITCH: And we see how humbling it is. Even on small tasks, you can be easily an order of magnitude of in terms of your estimates. And we do actually have a model for understanding this. It's called the Cone of Uncertainty. Who's heard of the Cone of Uncertainty? About half the room so time going left to right, it's basically your uncertainty around budget and schedule narrowing as you get further down to the project.
STUART LEITCH: And it's probably true in many industries but we like to cite a lot in software that you will actually know what the project is going to cost and how long it's going to take when you're done. And that is a bit of a problem. But related to this, what's interesting is that if you look at books around software estimation-- and they are kind of interesting books to read.
STUART LEITCH: There's no magic formulas in there, it's just basically how do you bring more and more attention to the estimation process. But one of the basic practices they talk about is bracketing with high and low estimates. And in part, this is designed such that you're actively actually thinking about the uncertainty. Well, what's interesting is that academia has been able to then go and study, though, that bracketing relative to actuals.
STUART LEITCH: And at least the studies that I looked at suggested that the bracketing actually wasn't wide enough, that the brackets were too narrow. So that's saying that even when we're explicitly trying to account for uncertainty, we have a hard time fully accepting the reality of it. And so the takeaway here is that we really need to explicitly manage what we don't know.
SPEAKER 2: "Understand that scope expands fractally with detail."
STUART LEITCH: So determining the length of the country's coastline sounds pretty straightforward, but it's not. The answer actually depends on the length of the rule that you use for the measurements. The shorter the ruler, the more bays and inlets you capture than if you had a larger ruler. And so as you essentially measure in more and more detail, the length of the coastline actually expands. And there truly is no answer to what's the length of the coastline.
STUART LEITCH: It actually depends on the measurement. And this applies equally to software requirements. As you get further and further down into the detail, it expands out fractally. And you just want to be really aware of that issue going in. And if you're going to be really prescriptive at a super detail level, you need to make sure that you've really allowed for that upfront or that you're not you're killing off scope somewhere else.
SPEAKER 2: "Requirements volatility must be managed."
STUART LEITCH: So as you get into a project, you should expect that your understanding of it is going to evolve. And that's likely going to drive changing requirements. That's actually a good thing. But it can also be a really, really toxic thing. And the challenge is that when you change requirements, it often introduces rework, it creating efficient, you're having to kind of go through a whole lot of communication to get people realigned around that.
STUART LEITCH: And you'll often have to go and cut scope elsewhere to have the project remain right sized. And depending on who your stakeholders are kind of in the middle of the process, you also need to really watch the tendency to try and perfect everything. As you get greater knowledge, you can always rework things, you can always come up with a better solution. And it's very easy for changing requirements to get so out of hand that they actually threaten the entire project.
STUART LEITCH: So the message here is that you want to be really disciplined around what you let change in a software project.
SPEAKER 2: "Estimates are probabilistic, but actuals are right skewed. So this needs to be factored in."
STUART LEITCH: So this is a slide from a few years back. This is on us household income distribution. And the first thing you can notice is that it's not a normal distribution, it's actually right skewed. And that kind of make sense. If you think about us as a wage earner, well, you're really earning zero dollars or above. So it's low bounded, but particularly when you think about stock traders or elite athletes, there's really there's no real upper bound here.
STUART LEITCH: And I mentioned yesterday-- well, actually yes, it was yesterday-- lies, damn lies, and statistics. Now this wasn't too impugn that statistics aren't a developed branch of mathematics. It was really to say that without careful treatment, it's very easy to draw the wrong conclusions. And if we were to use a single statistic to describe income on this curve, what would we use?
STUART LEITCH: If we thought about a random household, well, the most likely income they going to have is going to be between $15,000 and $20,000, because that's the mode. If we were to split the population in half, we'd have $50,000, that's the median. But then if we were to look at the average, well, the average is $72,000. And there's really not a right answer.
STUART LEITCH: In many respects, if you're actually working with these questions, it probably depends on what political party affiliation you have and what the agenda you're trying to drive is. But the point here is just basically thinking about different ways to describe these and the challenges that most people conceptually deal with averages, but their actual lived experience is with the thing that they most frequently experiencing, which is much more related to the mode and the median.
STUART LEITCH: And this ties into how we estimate. Because human beings tend to estimate based on their accumulation of direct experience. And so estimates actually tend to cluster around the median and towards the mode rather than the actual average. And if you ask your accountant what actually matters, well, he'll tell you it's the average. He doesn't care about the rest of this stuff. But when you've got a software team that's essentially estimating around, say, the median, you actually need to be able to account for that.
STUART LEITCH: And one of the things we've done at Silverchair is we have all of our internal data about our estimations and our actuals. And so we've actually been able to look at essentially the sum of the actuals relative to the sum of the estimates. And we do find that there's about a 30% delta there. And this actually happens for us to correspond approximately to the difference between the median and the average.
STUART LEITCH: And so the devil is in the detail with software development. It's very common for outliers to really drag the average to the right. And the purpose of an estimate is basically you're not looking for precision there. We know through the Cone of Uncertainty that if you really want precision, you've got to actually fully complete the work or accuracy I should say.
STUART LEITCH: The point is to basically get enough of a handle on the overall project that in aggregate it can be controllable. What we do is we periodically check in and do evidenced by estimation where we're looking at this and saying, OK, how far are we actually falling from our original estimates in terms of actuals? And then we just have simulators just make that adjustment. We don't make a big deal about it.
STUART LEITCH: We're just trying to recalibrate. So we're generally in the right ballpark.
SPEAKER 2: "Adding people to a late project makes it later because of the communication between nodes.
STUART LEITCH: So who has heard of this book? OK, quite a few people. Who's read the book? OK, a few people. Well this is a classic this is Fred Brooks from 1975, the year I was born. And it is filled with nuggets. So this was about Fred's time at IBM and got a bunch of timeless wisdom in it. But one of the headlining sites was that adding people to a late project makes it late.
STUART LEITCH: And the reason is primarily driven by communication overhead, which explodes exponentially as you add more people into a project. And he goes on to say that complex programming projects cannot be perfectly partitioned into discrete tasks without complex interrelationships. Now, if you listen to the folks with all the buzz around microservices, you would think that they've actually solved that problem.
STUART LEITCH: But if you speak to anybody that's actually done this at scale, they'll disabuse you of that notion. Microservices are helpful, but they're not a silver bullet. And there's kind of a consistent theme in software that there really are no silver bullets. And actually, that was Fred Brooks's next book, No Silver Bullet. So, go on to the next slide.
SPEAKER 2: How you organize will be driven by trade offs between specialization, communication overhead, and cycle time.
STUART LEITCH: So we just talked about communication overhead. But specialization it's really foundational to the productivity gains of the modern economy. Specialization by and large is a really good thing. But the challenges is as people become increasingly specialized, you then need to add more people in, which then places a lot of pressure on your communication overhead. And so you then essentially stop petitioning your work to create smaller teams.
STUART LEITCH: And that's where kind of things like microservices kick in. But the challenge is that we're in an increasingly rapid change environment within our businesses overall. And we see these cross cutting influences come in that actually require a lot of things to be changed across teams across services. And that's where you actually have to really think about cycle time, because microservices is definitely all the rage right now.
STUART LEITCH: But if you actually have to go and refactor across a whole bunch of microservices, you'll be asking yourself, how did the cycle time get so long? And that's actually the dance you have to do between these three trade off to really think about how change is likely to flow through your business and in what cycle time do you need to respond to that. And it was interesting what John Shaw said yesterday in the technology roundtable.
STUART LEITCH: Is John still here? No. But he was talking about how they are now requiring technologists to be able to interact with their customers. So they're narrowing down the labor pool that they can select from. But I suspect that it's really driving down their communication nodes and actually speeding up their cycle time.
STUART LEITCH: So that's a typical kind of trade that you have to think about.
SPEAKER 2: "Colocation versus distribution and culture matter a lot."
STUART LEITCH: So 200 years ago, if we all wanted to be in a room like this together, we'd basically have to get on our horses and kind of all come together. We've now got more modern modes of transportation, but we've also got fancy electronic communication where we could actually all be doing this virtually. And there's a lot of cost to get everybody in this room. So the question is, well, why didn't we just do this virtually?
STUART LEITCH: Well, that's a bit of a rhetorical question. There's actually a lot of intangibles here; number one, attention, like your actual engagement is a lot higher, there's the relationships that are being built, there's the relatedness, there's the conversations that are happening outside this room, not to mention the booze ups afterwards, which people rave about. The same dynamics are present within teams.
STUART LEITCH: In complex knowledge work, you're wanting to harness both the individual intelligence, but you're also wanting to harvest the collective intelligence. And that collective intelligence is really driven by the relatedness and the connectedness of your people. And so it's certainly possible to do this with remote teams.
STUART LEITCH: But like an event like this, it's a lot easier to do in person, so you just want to be really thoughtful of those dynamics. And related to this is the notion of power distance, or strength of social hierarchy. This is really about, does a person defer to authority, or do they challenge and upgrade perspectives coming from above? And as businesses are dealing with increasing change and complexity, traditional command and control management structures are really breaking down.
STUART LEITCH: And you want software developers that who are fully engaged in the process and not just order taking. And at Silverchair, we really think about this in terms of our hiring process. It's something that we explicitly screen for. We want people that can actually just push back and that are actually willing to do that, because it takes something to do that. Our labor pool is US-based, so it's actually a lot easier for us, because in terms of global averages, Americans tend to just-- we're a lot more willing to speak truth to power.
STUART LEITCH: Those of you that have offshore in India or other locations, you'll probably have the experience where they can go very fast in a particular direction. But if you're not really clear in the right direction, they're not going to tell you. And so this is just something to think about that it's a trap if you're not actively managing it when offshoring, because you want people that will challenge the plan when they say misalignment with the overall intent.
SPEAKER 2: "Agile is largely a capitulation to uncertainty in the face of complexity but is pragmatic if you don't buy into the hype."
STUART LEITCH: So this is basically a chart. And I shared this a couple of days ago with a few folks. There are a few overlapping slides here. But software development is basically about 50 years old. We say agile methodologies pop up in 1995 through 2000. Agile really only became a thing in 2001 with the Agile Manifesto. But we often hear about agile in contrast to waterfall. Well, that's a bit of a misnomer.
STUART LEITCH: It's that the methodologies from the 80s onwards actually had spiral or iterative dynamics within them. It was more that there were just so heavy weight. And there was a real breakdown in terms of the long range planning that was involved to actually produce predictable results. And agile was largely a rebellion against that. And what it did was explicitly optimized for change. It really shifted the posture and essentially said, OK, we're just going to operate in very short iterations, and we're actually not going to-- at least in terms of something like Scrum-- we're not explicitly going to even try and plan beyond a sprint.
STUART LEITCH: And that worked very well for small consulting teams and product teams. But it didn't work well at scale. And so we've seen an evolution of methodologies here that have added in more and more weight. And there's certainly been a lot of the weight from prior frameworks that slipped in here. And each framework will be sold to you as the next best thing since sliced bread.
STUART LEITCH: And there's typically a lot of fear around this. You'll see articles, are you doing agile right? There is no silver bullet. And these are really just guides that you need to select from and think contextually about what fits for you. This is the most popular framework for larger organizations, this is scaled agile. And this is starting to look pretty complex.
STUART LEITCH: How we at Silverchair look at this is we look at each of the prescriptions here and we ask ourselves, what's the intent of that prescription, how does that fit in our particular context? Because software development environments are complex, it's not one size fits all. And then we ask ourselves, does this prescription add decisive value for us? And ultimately, we use the adage, as much as necessary, as little as possible, because you have to keep pruning your process or you will calcify as a business.
STUART LEITCH: And these are the values that were the original Agile Manifesto. There's a little bit more to it. But you ask anybody that was there, and this is basically what actually mattered. And you want to keep these in mind if you're doing software development, you've got agile consultants in, because what they tend to sell is the processes, the tools, and the planning devices, because that's part of what they can sell.
STUART LEITCH: But this is actually really the essence of agile. And another way of looking at this is to think of agile really as a tight feedback loop. All of these processes are trying to facilitate throughout the organization people just being in a learning organization relative to the work that they're doing.
SPEAKER 2: "Schedule compression is a killer and iteratively MVP, if you can."
STUART LEITCH: So we talked earlier about the communication overhead that occurs and when you are trying to think about, well, how aggressive should I be with my project schedule here? Theory would tell us that if we get too aggressive with it, the overall relative effort that we incur is going to kind of hockey stick up there. And likewise, we also have this notion of Parkinson's law. Who knows Parkinson's law? Well, basically, work expands to fill the time allotted.
STUART LEITCH: And there's been some studies done on this. And this is a meta study that is essentially looking at these. And what it's showing is that schedule compression across these studies was absolutely found to be true. You can really compress schedule, but you're going to have to do it by throwing a lot of extra dollars in there-- inconclusive about Parkinson's law.
STUART LEITCH: MVP, MVP has now become a bit of a dirty word. It kind of is icky because sometimes people call something an MVP where it actually kind of really isn't. But if you've got the architecture and the business context that will support iterative development, it's generally a better way to go.
SPEAKER 2: "If you want to move fast, you must be willing to break things or spend a lot of money."
STUART LEITCH: So we typically want both speed and safety. But getting more of one typically comes at the expense of the other. Building in safety processes tends to slow things down. But it does reduce risk. And we need to be continually challenging the assumption that we've actually got the risk right and really understand what our risk tolerance is for the particular aspect of the business that's being impacted by the software development that we're working on.
STUART LEITCH: Now Formula 1 is one of those interesting kind of exceptions here where-- this is a crash obviously. But in general, relative to how fast they go, I'm actually surprised that there aren't more fatalities. It actually is relatively safe. Well, I actually did check with this photo, and they all walked away. They were actually unhurt, no hospitalization, nothing.
STUART LEITCH: That's pretty crazy. But it also costs. The cheapest team in Formula 1 is spending about $100 million a year. To be competitive, you apparently need to spend about 250 million a year. So there are real trade offs here. And most people are going to essentially find some balance between speed and safety.
STUART LEITCH: Now I'm going to kind of tell a personal anecdote here about my run in with software speed and safety. So this is an electric unicycle. And these are kind of marvelous things. There are an evolution of the Segway but just with one wheel. And it's like a briefcase with pedals on the side and a wheel in it. And you basically stand on it. And you have to balance it side to side, like a bicycle.
STUART LEITCH: And it sounds really hard, but it's actually just like riding a bicycle. But you then have to let go of your balance forward to back. And you basically fall forward. It detects that and it applies torque and gets underneath you. And then you fall back to slow down, remarkable thing. And one of my work colleagues got one of these. And he was zipping around on it, oh, it's so awesome, all this other stuff.
STUART LEITCH: And I was just like, whatever. Anyway, he has this accident on it, and he breaks his arm. And I'm like, bet you're not going to be doing that anymore. And he's like, oh no, I was really serious I'm just telling you how awesome this thing is. It's like a magic carpet. I mean, I go everywhere in Charlottesville on it. And so I learned more about it. And I learned that he actually had you know an older model that wasn't that sophisticated and actually used within it technology out of electric bicycle.
STUART LEITCH: And one of the things in that technology is that it's got overvoltage protection, just to really protect the longevity of your bike so that you don't wear out the components too soon. But the challenge with these wheels is that when you're riding along, you're typically going really fast. And you might hit like a rock on the road. And so that immediately pitches you forward.
STUART LEITCH: So the wheel has to surge with power to basically get underneath you. But what would happen with these bicycle components was it would detect the overvoltage and it would shut off the power. And it's even worse than it sounds, because the motors no longer actually a motor, it's a generator. But it's got no way to put the energy.
STUART LEITCH: So basically, the electricity can't go any way, so the wheel just basically immediately locks up. And it's got a fly swatter effect. And so it is actually pretty dangerous, but I was intrigued by this. And I went out, and I got myself a wheel. I got a current generation one, I got one that was really pretty safe. And I ended up putting over 1,000 miles on it zipping all around Charlottesville.
STUART LEITCH: And I actually stopped using my car to such an extent that I had to schedule it in my calendar to go and stop the thing so the battery wouldn't run down. Ultimately sold the car. And when I sold the car, I thought well I just need to get the best in the world at this, I have to get the best. And so it then became a whole mission to figure out, well, what was the best?
STUART LEITCH: And this is where I had this trade off to make, that there were two basic brands. They were both Chinese brands and this is in part because they just kind of relentlessly rip off the intellectual property of the US companies. So it's now just these Chinese companies. But you've got these brands that are known for different things. And King Song is essentially known for being much more conservative, and they're really safety first.
STUART LEITCH: And Gotway, on the other hand, they're just like sports cars, they push the envelope. And I was looking at this and trying to understand the differences. And one of the things-- they were both priced about the same, but Gotway was introducing a higher voltage. And higher voltage means more energy at the same current so you can squeeze more out of your wheel.
STUART LEITCH: And the thing that really jumped out to me, this one goes like 20 something miles an hour, this goes 30 miles an hour. And yet, Gotway actually had a reputation for being a little less conservative on the safety side. But I ended up deciding that I was actually willing to run that risk. And so I kind of signed up.
STUART LEITCH: They were releasing both these models. It just so happened to be that I was quite lucky. I missed out on the first shipment of products coming over from China, and I got a second batch. But while the second batch was on the way, all the people trying out the first batch, well, I started to have the fly swatter thing happen to them. And if you kind of Google the images around that time, it's pretty nasty, people like with teeth missing and all.
STUART LEITCH: But anyway, it turns out this was a firmware issue. They'd done all the prototyping, and it really worked, and they were just kind of fine tuning this stuff. And they just made some mistakes in the software. And this happens. They were rushing to market. And as a result of that, they just intercepted mine as it came out of customs in Los Angeles, they flashed the firmware.
STUART LEITCH: I've had it for three years, I love the thing. I've ridden over 3,000 miles on it now. But this is a really practical example. It's not just hypothetical that you actually have to really think about the speed versus safety dimensions. And a lot of you are running a lot of money over various platforms, and you really need to think about which dimensions of that are more susceptible to-- what's the impact of business disruption if aspects of that goes down?
SPEAKER 2: "Be pragmatic about the long tail of edge case defects, and don't sweat them." So any sufficiently complex software system has a very long tail of defects that are not economically viable to fix. We often don't talk about that, but it's just real. And closing issues as won't fix is common practice for all major software vendors. And I'm just going to give you two examples of this. So this is essentially, we're looking at Firefox here.
SPEAKER 2: Firefox is a great browser, very stable. Probably half the people in this room use it. The reason I picked Firefox is that their bug databases public, so we can essentially look at that. This is filtering down and saying, OK, this is just looking at defects for Firefox saying in the last 30 days, there were 119 defects closed as won't fix, that's how it works. This is Atlassian.
SPEAKER 2: We use the Jira and Confluence products. Specifically looking at Jira, in the last 30 days, over 200 bugs closed just won't fix. The point here is we all want quality. But we need to be really thoughtful about what we're pursuing in terms of quality, because there's definitely a trade off. And if you're sufficiently close to the product and you are looking for perfection, you're going to be disappointed, because you actually can't get all the long tail of defects out there.
SPEAKER 2: So you want to take a really blended approach. And you want to be really cautious of the OCD types that are involved in the project, because they can really wrap things around the axle. You're thinking about, what are the issues that actually really drive value versus the things that just drive perfection? "Try to think in terms of long term cost of ownership and hew to the happy path, deviating with customization for decisive value drivers."
STUART LEITCH: So we've all seen a slide like this. This is the concept of an iceberg. And the challenges that we know that there's a lot of stuff under the iceberg. We actually know that's where the bulk of its mass is, but we only say the thing at the top. And this is something to really think about when we customize software. And generic advice that I give to people is to think about what the happy path of that software is.
STUART LEITCH: How does this software actually want to work? And really try and map that onto your business needs and say, can we actually-- you don't want to just take some piece of software and radically adapt it so that it fits your world. You actually need to be willing to adapt to it as well, because the further you get off that happy path, you're going to start to see gaps open up in the user experience.
STUART LEITCH: You're going to see it become increasingly clunky. And you're going to spend a lot of money doing the configuration and the customization to get it there. And you're going to find more and more things break in the future. And so there's a real cost to customization that people often don't really own. And the point is customization should really be focused for the things that add decisive business value.
SPEAKER 2: "Finally, effective engagement of knowledge workers is what outweighs all else."
STUART LEITCH: So if agile is about decentralizing decision making and creating tight feedback loops, it's decisive that hearts and minds of those people who are actually in that process are aligned. And you want to get your workplace beyond a transactional environment when people have real heart in the game and experience enough psychological safety that allows them to show up with commitment and actively cause effectiveness in the feedback loops around them.
SPEAKER 3: "Software development is an industry with rampant cost and schedule overruns. Its high complexity means we need to explicitly manage what we don't know, understand that scope expands fractally with detail and that requirements volatility must be managed. Estimates are probabilistic, but actuals are right skewed, so this needs to be factored in. Adding people to a late project makes it later because of the communication between nodes.
SPEAKER 3: How you organize will be driven by trade offs between specialization, communication overhead, and cycle time. Colocation versus distribution and culture matter a lot. Agile is light largely a capitulation to uncertainty in the face of complexity but is pragmatic if you don't buy into the hype. Schedule compression is a killer and iteratively MVP, if you can.
SPEAKER 3: If you want to move fast, you must be willing to break things or spend a lot of money. Be pragmatic about the long tail of edge case defects, and don't sweat them. Try to think in terms of long term cost of ownership and hue to the happy path, deviating with customization for decisive value drivers. Finally, effective engagement of knowledge workers is what outweighs all else."
STUART LEITCH: Wow, I was almost overcome with cognitive dissonance. It is challenging, isn't it? Having somebody raid and then you're kind of reading ahead and you have to keep aligning with that. Well, I wanted to just get some feedback as to the pain of that, was that outweighed by the effectiveness? So I'd like to see a show of hands-- and just be honest-- do you think that this helped with retention of information and comprehension?
STUART LEITCH: OK, yeah, about 3/4 of the room. OK, well, fascinating. Well, I'll end this by saying I've shared you know some context that I've experienced through my career. But I actually need to apply to what I've said to you to my critique of almost everything in this industry. And that is that no rules are universal, maybe this one's universal.
STUART LEITCH: But no rules are universal, and all rules need context. And you really need to be continuing to try on what's out there and discard things that are no longer serving you. So thank you very much. I think we're pretty much at time
SPEAKER 3: I think we can take one question, one question. So we have time for one question. Anyone out there have a question for Stuart?
STUART LEITCH: I was sure somebody was going to ask me a unicycle question
AUDIENCE: Do you have one of those or not?
STUART LEITCH: I do. I actually went got a full face helmet. I actually had this conversation about was I willing to live with a busted up face. And I decided I'd just wear a full face helmet.
TIM BARTON: So I'm Tim Barton, I'm Stuart's excellent boss. sure I think people in the room, there may be people from large companies that do software developing themselves and then other models that you outsource. Have you got anything to say that might be useful to people in the room who are in the outsourcing side of side of things in terms of when they're not doing the software development themselves, but they're in a relationship with an organization that's doing software development for them, is there anything you'd say that would help them think about that?
STUART LEITCH: Yeah, I think software development is rarely a set it and forget it exercise. Even if you do big requirements planning up front, one of the things that we kind of see with the evolution of agile is that stuff never really worked all that well to begin with. And so you are going to have changing requirements. You are going to hit up against unknowns that are going to derail your project. And you're going to have to basically collectively get in there and wrestle this thing back onto the tracks.
STUART LEITCH: And so you want to be thinking about who are you actually going to be doing that with. Are you going to be doing that with a partner, or are you just going to be doing that in a traditional vendor relationship? Because it makes a remarkable difference what the relationship actually is.
SPEAKER 4: Thank you, Stuart and thank you, Susan.
Form Title Download
Transcript
Form Title Bookmark
Entire Media
Segment(s)
Custom Clip
Start At:
End At:
Form Title Share
Entire Media
Segment(s)
Custom Clip
Start At:
End At:
Form Title Accessibility and Keyboard Shortcuts
The cadmore media player is a professional video and audio player experience designed to deliver media as well as relevant transcript, segmentation, and metadata. On the screen is a traditional html video player with buttons such as play, pause, forward, captioning, language and play speed selection and more. Adjacent to the video player are generally tabs, one of which shows a scrolling transcript of the video and the other shows any segmentation of the media, sometimes referred to as chapters. At the top of the video is a menu bar containing an Explore button and Tools button. The Explore button allows you to dig deeper into the video by opening up an explore dialog which contains tabs for a visual navigation of the video using thumbnails, an about tab which contains rich metadata about the media, a segments tab which contains further information about the video, and a related tab which contains links to other media related to this media. The tools button opens a menu with options for sharing the media on social media, creating bookmarking links, creating embed codes for your website, generating citations, and more.
Keyboard Shortcuts
Keyboard shortcuts only work while you are in forms mode. By entering forms mode, you will be able to quickly access portions of the media player with single key commands.
Spacebar: Play/Pause Toggle
I: Info Menu,
O: Tools Menu
T: Transcript Tab,
S: Segments Tab
F: Forward 15,
R: Rewind 15
L: Languages,
P: Playspeed
M: Mute Toggle,
V: Volume Bar
N: Video Only Toggle,
B: Full Screen Toggle
C: Captions Toggle
Escape: Closes dialogs and menus / Exits Player
Z: Next Transcript Chapter
X: Previous Transcript Chapter
Q: Next Transcript Paragraph
W: Previous Transcript Paragraph
Entire Media
Segment(s)
Custom Clip
Start At:
End At:
Embed Size
-
You Size - Responsive
-
We Size - Responsive
-
Picture Overlay Popup
-
400 x 225
-
512 x 288
-
400 x 700
-
768 x 1024
-
1024 x 512
-
1280 x 608
Form Title Cite
No citation provided.
Info
Segments
Related
Thumbnails
Exit Clip Mode
You have requested to see a portion of the video outside of the clip.
Click OK to switch to the full video or click if you would like to stay inside the clip. You can always go back to the clip with Tools and Clip Mode