How long does it take for adding an InputBox?
When the Product Owner told you it’s a small change, don’t trust him.
An estimation session
After the iteration planning meeting, when all the stories had been walked through, the PO turned to you with a lovely smile, pretending it was just an impromptu chat, asking you how long would it take for adding an InputBox to one of the pages.
What he described was a “simple” InputBox for the user to enter his/her address along with other personal information and persist to the backend.
He even sketched an irregular box on a whiteboard to illustrate the idea and trying to make it more clear to you. He looked at you and asked,
“How long do you think it will take, is that even worthwhile to estimate it and put it into the backlog, or can we just do it along with some other small card on the wall?”
Your gut feeling told you something important was missing, but you were not too sure what exactly it was.
You reckoned for a little while, and told the PO “It might take, ideally, 5 to 6 days, and if we consider meetings or other activities that would be longer…”
“A week for just an InputBox?”, knocking the irregular box on the whiteboard, the PO looked quite shocked, you could even hear the saved half “have you lost your mind?”.
“Well, I’m saying, ideally, normally that will be longer in practice…”
“……”
A significant gap like this usually means people are comparing apple to an orange, it either the product owner is over-simplifying the problem, or the developer is exaggerating it.
The missed details
Because of knowledge barriers and oversimplifying without drill down deep enough into the details, we normally would underestimate things without knowing it. Additionally, if we ignore the resistance in the outside environment, we might easily make mistakes in terms of estimation.
A simple InputBox
From the PO’s point of view, a simple InputBox might be something looks like this:
A user would type some text and save it to the backend. Of course, some validation rules — such as the text can not be too long or too short — are needed.
A not so simple InputBox
However, from an experienced developer’s perspective, that’s totally a different story. A simple InputBox means something like:
Obviously it has more states thus it is more complicated, some of the states might include:
- disabled
- empty
- focused
- invalid
- with helperText
- accessibility
Generally speaking, in the initial state, the InputBox shows a placeholder, when the user starts to typing, a lot of realtime feedback are needed: spelling check, length check, format and so on. Additionally, other parts of the page may have an impact on current InputBox as well. For example, an unauthorised user is not supposed to be able to type anything, so we have to disable it.
Different states mean different UI styles, such as font-size, font-family, colour, margin and padding, etc. Those trivial details would take many back and forth time, which is easily to be neglected by most of us during the estimation stage.
Another time-consuming work is validation (and define the rules, again, it’s rarely seen the rules can be identified clearly by just one round).
Validation
Validation, as a cross-cutting requirement, is always being underestimated in projects. Apart from the basic validation rules like length, email address, normally there will be some complicated rules that potentially would break the existing implementation logic.
For instance, when developers came up with a solution for validation like:
const validations = {
minLength: 1,
maxLength: 16,
}<AddressSearch validations={validations} value={value} />
After some rounds of debugging and refactoring, it works well in the way like:
const builtIns = {
minLength: (value, criteria) => value && value.length > criteria,
maxLength: (value, criteria) => value && value.length < criteria
}const isValid = (validations) => (value) => {
return _.every(validations, (k, v) => builtIns[k](value, v))
}const AddressSearch = ({validations, value}) => (<Input error={isValid(validations)(value)} ... />);
All of a sudden, the next requirement is connect the validation of AddressSearch to another InputBox: when the value in the country Dropdown changed, rules for AddressSearch are changed correspondingly.
Changes like this would break some logic in a lot of places, and the developer might need more time to apply the changes across the application, and tests respectively. Code like builtIns in the snippet shows above has to be rewritten.
Restrictions
Along with validation, for some fields, we want to stop a user from typing some of the characters (like alphabet only, no whitespace, etc.). Sometimes I call it restrictions, and it can be seen as an extension of validation.
Some examples for restrictions could be, as an InputBox:
- No special characters allowed
- Only digits
- Only alphabets allowed
- 1–12 as months or 1–31 as days
- Digits and dot
- Sometimes those rules are orthogonal, but sometimes they might interfere with each other.
For example, you want to collect the phone number by using a InputBox like:
<input type="number" />However, there is a defect when a user has to provide prefix 0 for state code or something else (or maybe +60). And the browser is smart enough to remove the leading 0 for you.
You could do this to get it fixed.
<input type="tel" pattern="[0-9]*">But new issues may be emerging later on. In summary, every potential issue and its corresponding solution has their own problems, and it’s normally very difficult for us to foresee those efforts.
Even for experienced developers it is difficult, for people playing other roles in the team, situation can be even worse.
One more variable
Now I believe the PO got a better understanding of how much efforts are needed for a simple InputBox. That's just a small piece of the iceberg.
Let’s assume that we need to enhance the InputBox a little more: when a user typing address, we need to search the snippet and auto-complete a full address matched.
Whenever the network is introduced into a system, things turn to be much more complex, and the worse part is the impaction is not linear. On the one hand, the async system calling itself is more unreliable and complicated than you consume local data. On the other hand, there are too many variables that out of your control: network speed, network failure (routing, firewall, etc.).
Additionally, once you have frontend and backend separation, you need to work out the protocol between both ends. How do you make sure the both sides are synced all the time?
It’s prevalent to see that UI need something and eventually found that field is not available from backend, or backend saving date as long but frontend needs timezone as metadata or the like.
The same rule applies to validation too. You need logic both in UI forms and in backend models and persistence layer. When you’re using isomorphism architecture, you might use the same piece of code for both backend and frontend that could relieve the pain. But it’s hard for cases that using different languages in UI and backend — you have to put the same logic in both languages, on both sides. And the problem is, again, how do you make sure those changes happened in synchronised?
Of course, I believe from the technical perspective, and we can eventually find some solutions and some trade-offs for those problems. But you should keep in mind that those things don’t happen automatically or for free, they come with some cost.
Even for experienced developers and members who have the willingness to keep learning, it still remains true in most cases. After all, a lot of problems cannot foresee, and surely you cannot resolve anything you don’t know yet.
The happy path pitfall
In most cases, people tend to think things in the logic way, the happy path, include estimation as well. But in the development stage, the so-called happy-paths are actually rare, they are abnormal. Many factors in the real world can break our application in unexpected ways.
Network timeout, service downtime, routinely backend service upgrade, incompatible browsers, unmatched operation system, different hardware, and so on. All of those could cause failures in the application level.
Since it’s inevitable for failing, it seems the only option for us as developers is to design some conceive protection mechanisms. Firstly, we need something to recovery the application from a failure (instead of a blank page). Secondly, we need to record some error information for us to debug or help us to do the further investigation.
Cross functional requirements
Apart from functionality, there are other factors to be considered, as well. Such as accessibility and usability (like wording on the page to make the experience more smooth), and compatible as well.
Some common cross-functional requirements are:
- Cross browsers supporting
- Multiple devices supporting
- *blities
Another thing people keep ignoring is for responsive design, Dev / BA has to work closely with UX to make sure the design works well on different screens. In most scenarios, the way how a user interacts in a tablet is totally different from that in desktop, and font-size, animations too.
And I’ve seen in many projects the assumption for adopt pages to smaller screens is just adjust the font a little bigger. Unfortunately, it is not true in most scenarios.
Beyond technical details
Besides the unpredictable latency and barrier from the technical perspective, there are some other more time-consuming factors. They are everywhere, for each one it’s just trivial and tiny, but in accumulation, the effort can be significant.
Chaos is normal
Let’s say in a project there are several Devs, let’s put the names into an array:
const roster = [
'Matt C',
'Santosh',
'Patricia',
'Nicole',
'Juntao'
]Generally speaking, everybody has to handle something in daily life, such as holiday, sick leave, traffic delay, or lost on the way for coffee.
const execuses = [
'is stuck in traffic',
'is running later',
'has to pick up kid',
'is not feeling very well, take half a day off',
'is having a holiday in Hawaii'
]And life is exactly like the code below:
`${_.shuffle(roster)[0]} ${_.shuffle(execuses)[0]}`That’s why, in the real world, you would see something like this happens usually:
Maybe it’s unlikely to happen each day, but in the relatively long run, you could see it happen almost certainly. And it could be worth if there are enough parties in a project, when the team gets bigger, the possibility of uncertainty increase as well. At some point then, everything is not working is normal, after all, that’s the world we’re living in.
All of those uncertainties could cause our estimation to fail, and surely we cannot see them at the very beginning. Thus we underestimate.
A little story
Several years ago, in an estimation session, I gave my estimation as three days for a RESTful API of a resource, and the tech lead from our client looked pretty unsatisfied by that. He told me he could finish it in half a day - it was just a CURD after all.
I helped him to list some of the tasks, and he mused for a few minutes and agreed my estimation:
- Database migration
- Entity definition
- Relationship between Entities
- Service / Controller
- Exception handling
- Unit tests
- The pact between our system to downstream
- Integration tests
As far as I could recall, it took at least three days, actually just found the right people we can get information from took a whole day.
How can we handle that
I guess you have already got some of the ideas about how we underestimate ahead of the actual work. To handle this problem, fundamental principle, of course, is to do exactly opposite as what we would normally do it. In practice, a feasible solution is to break the work down to small pieces that don’t have too much uncertainty, and discovery more details that we could not see in higher level.
Such as different states of an InputBox, the transitions between states, styles for each state, and so on could be addressed by elaboration. During the elaboration, details start to emerge by themselves. Furthermore, when you have to deal with data fetching, loading, loading failure, no data available are all come and then can drive more uncharted details, which potentially will make the estimation more accurate.
A state machine can be used as a manner to enumerate all the possible states of a component or a group of components, that can reveal potentially all the combination of states and all the variations.
Additionally, in your heart, you have to admit the world is complicated. In a large organisation, things are all mingled together in a super complex way, all of those factors can affect each other and in the surface things looks rather unpredictable. Instead of resist it, we have to embrace the changes proactively. On the other hand, it drives us to adopt the simpler design, employ solid infrastructure, utilise high-quality build methodologies, and so on to handle the overall complexity.
Summary
At this point, I suppose you already know what’s happened in delivering the InputBox, and yes, although there are some buffer in the estimation, it still took more than one week to get it done.
Because of the knowledge barrier and biases, we tend to ignore details which actually can impact the effort a lot. Furthermore, we are in a world that is super complicated and full of uncertainties in many ways, different factors are interconnected with each other, and it’s easy for us to loss the objective view to evaluate. If you ignore the uncertainty, you underestimate the actual effort significantly.
One way to reduce the complexity is elaboration, thus improve reliability. The more you break down, the accurate the estimation will be (of course, the break down itself IS efforts as well). On the other hand, we need to embrace the chaos and build software in an easy-to-change and straightforward manner to make it more flexible and adjustable.
P.S. I planned to write this article in around three days, just drawing the mockups took me two nights. Drafting and spelling check took me another two, and one extra for polishing the language.
