Working in a corporate environment is loads of fun and oftentimes it feels more like an adult kindergarten than a serious work environment. The clowns will always make sure to keep the rest of the employees entertained. First thing that happens in the morning is the arrival of some overly dramatic email that is complaining about yesterday’s mayhem and it usually goes something like this:
We found out blah blah blah yawn, that the dirty dishes in the sink are the real problem, because we might bring the customers here and what will they think about the sink full of dirty plates blah blah blah and therefore we pronounce the hotshots from office A as great as they keep the work environment clean, but the office B is full of pigs because they keep making this mess.
From now on, if the dishwasher is full it’s your duty to wash the dirty dishes after use. We will have a meeting about this problem today, please send us any suggestions before noon at such and such email.
I can already imagine how much fun such impromptu gatherings are. A dozen or so bored middle managers sitting around the table, writing emails about other things as the meeting roams forward. Somehow the more people there are in such meeting, the worse the final solution will be. If you ever sat on a meeting with more than 3 people present, you know that it is doomed to fail and nothing will ever get done.
The C something O at the head of the table starts the meeting: “Gentlemen. As you may know this pile of dirty dishes in the sink is a real problem in this office. We have to do something about it. May I ask Carl on my left to start presenting the possible solutions.”
Carl: Yes, thank you BIGSHOT. I am deeply concerned about this problem as well, so I tried to brainstorm the possible solutions. One would be to set up a camera that would detect the person who put a dirty dish into the sink instead of washing it manually. This will ensure that the trouble makers will not be able to hide in the crowd, and we won’t have to hire additional people just to deal with the pile.
BIGSHOT: thank you Carl, that is an excellent solution. Any objections? No? Okay we’ll define the details on the meeting next Monday. See you all there.
The crowd goes wild.
Interesting, but they are actually solving the wrong problemzzzzzz zzzzz.
Sloth! My manager wakes me up as he storms into my office. As you may know, I have an important meeting with the customer so I won’t be able to attend this dirty sink meeting. I count on you as my replacement to be present there and help them with resolving this problem for good. (Note to self: always tail logs on the big monitor so it doesn’t look like I am dozing off).
Oh great. My imagination served me well and this is really happening. But wait, aren’t they dealing with the same problems as we do? This is clearly the problem of scale that keeps bringing my servers down. Considering it was a slow day in the office, I made a quick back of the envelope calculation.
If we take some queuing theory in consideration and look at this utilization formula 1:
Wait time in queue (time unit) = Busy(%) / Idle(%)
Plotting this formula on the graph will show us the real nature of the beast. As the resource gets busier, the wait time for the tasks in their queue goes through the roof.
Calculating the wait time of the dishes in the sink for both offices is a breeze:
Office A (X people) ---------------------------- dishwasher busy: 20% dishwasher idle: 80% wait time = 20% / 80% = 0.25 Office B (3X people) ---------------------------- dishwasher busy: 60% (3X*20%) dishwasher idle: 40% wait time = 60% / 40% = 1.5 Wait time in queue difference ---------------------------- 1.5 / 0.25 = 6
In other words: the dirty plates that will start piling up in the sink of the office B will wait 6 times longer in the sink, than the dirty plates in the sink of the office A. These people are trying to solve the wrong problem and will gladly sit on a long and unproductive meeting due to their blind ideology!
So all things considered, I wrote an email to the person in charge of the cause:
Dear so and so,
I am afraid that you are solving the wrong problem because of the following reasons:
The real problem is the scale that the two dishwashers are facing. Given the same dishwasher and according to the provided calculations, it’s 6 times more likely that the pile will start forming in the sink of the office B as it has 3 times as many people in comparison to office A.
As you can see, the entire cycle of putting plates into the dishwasher, running it and unloading it is called a lead time. If you want to get rid of the pile in the sink, you have to lower the lead time. You can do that in two ways:
Lower the processing time by decreasing the time interval that the dishwasher takes to clean the dishes (e.g: from 30 minutes to 15 minutes).
Lower the buffering time - the amount of time the dishes stay in the sink before they are washed.
Chances are, you won’t be able to tweak the processing time much, as that is more or less defined by the washing machine vendor. Nevertheless the buffering time could be lowered tremendously.
The solution to lower the buffering time is simple but not quite intuitive. You have to reduce the batch size and wash dishes more often, even if the dishwasher is less full or not entirely full. In other words don’t start the dishwasher only when it’s full, start it as soon as it is economically viable.
This will have the following effect:
There will be less dishes in the dishwasher, so you can use a lower washing interval. Your lead time decreases accordingly.
There will be less rework - poorly washed plates that have to go back into the dishwasher and keep the honest first time dirty dishes waiting longer in the sink.
Since the dishwasher will be washing more often, the buffering time is decreased as well, which leads to a shorter lead time again.
With a long enough lead time the pile of dishes will just keep growing until you can no longer close the kitchen doors.
But, with a short lead time? There won’t be any pile.
Improvements made anywhere else besides the constraint are an illusion. In order to solve this problem you have to exploit the constraint (dishwasher) by running it as often as possible while keeping its lead time small 2.
Yours truly, Sloth
The meeting never took place, and I haven’t seen the full sink again. Queuing theory works and I am still alive.
I will gladly take the spotlight in order to bring another problem to light. Most middle managers still don’t quite understand the problem of having your engineers fully utilized. How can anybody argue that people working less will turn out results faster?
The answer to that is simple, take a look at the utilization graph. If your engineers are busy all the time, any unexpected event that might happen will wait in their work queue forever; as defined by the utilization formula 100/0 = ∞. They will never have the time to work on an unexpected problems, because the ongoing tasks are eating up their entire time.
In other words, if your kitchen sink is full of dirty dishes and everybody is busy all the time, that pile in the sink will just keep growing and growing and every new task will take forever before it’s completed. That’s not because that particular task would be hard or complicated, but it is mostly due to spending the majority of its time in the work queue of your engineers.
If you can reserve some of their time for improvements, then they will be able to tackle the unexpected problems faster or work on more important problems that will prevent the future ones before they even happen. Keep your sink clean or you will drown in dirty dishes when you least expect them.
Utilization formula is taken from a well known The Devops Handbook book. Despite the fact that nobody knows where it came from (the authors speculate it’s a simplified version of Little’s Law), it most definitely works as promised. I’ve seen it working when dealing with programming problems and I’ve seen it working outside of programming problems. ↩︎
But wouldn’t according to utilization formula the dishwasher being busy 100% cause infinite wait time for the dishes in the sink?
If the dishwasher is running all the time, that doesn’t mean it’s 100% utilized. If more dishes come into the sink, you just put more of them into the dishwasher. As long as the throughput of the dishwasher is greater than the rate at which the dirty dishes are arriving, everything is going to be fine. When the dirty dishes arrive faster than your dishwasher is able to wash them, then the pile will start forming and wait time of your queue becomes important. ↩︎