Eventually consistent world-view
Posted by: Manas in Uncategorized on December 19th, 2009
Till sometime back, we used to build systems that were consistent at all times. When the user takes an action, all the data will be updated at all places synchronously. If any of the step fails, the entire transaction will be rolled back so that there was no inconsistency in the data.
That’s how it was till sometime back.
Then came a time when the data became geographically distributed, more data had to be manipulated per request, the data resided in multiple repositories and most importantly, the response time had to be reduced. This led to a new class of design where the data wasn’t always consistent. It was eventually consistent.
Eventual consistency of data meant that you could update part of the data now, and part of the data later and some part much later. The reason for doing this was simple, people demanded more real-timeliness; more than was possible by keeping data always consistent. In the new scheme of things, the application design will try to keep data always consistent for a specific user while at the aggregate level, it may not be. And more importantly, even for a user, a glitch once in a while has become OK.
The users today have become more tolerant to once-in-a-while glitches while the responsiveness rules.
Now, let’s look at media. Once upon a time, the journalists used to work very hard to come up with a consistent world-view that they could present to their readers. They’d verify the information from multiple sources to ensure that there were no holes in the story. Of course, it was a time-consuming process. It required work that had to be done before the news was broken.
But the very nature of news required speed in breaking the news. This speed requirement conflicted with the operating model of corroborating the information from multiple sources or verifying things to ensure that whatever was reported was always consistent with what actually happened. The competition forced media to take a route where the news would be broken as soon as it arrives and further verification would be done a little later.
That’s where things became interesting. Now, there was a shift in the model. The responsiveness of a media setup became more important than accuracy. You could be wrong once-in-a-while but you had to be first. Earlier, a witness would tell a journalist and a journalist would in turn verify and publish. But now, since the journalist would just publish and verification might happen later, the witness could just short-circuit the whole process and directly publish.
The first wave of this transition happened with blogs and twitter now rules the scene. It’s all because of responsiveness. It’s true that people can post false information also. But then the right information also trickles along. People may be misled for a while but they are fine with that. They eventually get to know the truth. The world-view with Twitter is eventually consistent. And that’s all that is required.
Media may be in the denial mode for now but they should understand it very well. They were the ones who invented eventual consistency of world-view to increase responsiveness. Or rather, the world-view of people has always been eventually-consistent, the current media setup was the first one to leverage it for increasing their responsiveness.
Why Twitter Search can rival Google Search
Posted by: Manas in Uncategorized on June 17th, 2009
There is something about Twitter search that makes people ga-ga about it. It’s called real-timeliness. While Google search shows what’s been around for a longish time (in the internet sense of the word), Twitter can show you what’s the latest, right now kind-of-stuff.
However, one point is completely being missed out in the debate of aged-content Vs fresh-content. And the point is that PageRank algorithm, when applied to Twitter content, will give better results as compared to when it is applied to whole of web.
Let me explain.
PageRank algorithm essentially computes the popularity of a piece of content in the context of some keywords. So, there are two aspects: popularity and the context of keyword. Let me take them one by one:
- Popularity: It is measured by finding out how many other pages link to a particular page. Not all the links are considered equal. If a link comes from a domain that itself has a lot of incoming links, it is considered far more valuable compared to the domain which doesn’t have much of incoming links. This is in effect a mimickry of the reputation system that’s prevalent in the society. If a highly reputed person praises you, that praise is worth more than the praise from a less reputed person. And the reputation is a function of how much praise you eventually get.
- Keyword Context: So, when a link is made to a webpage, it is made within a context. That’s the keyword context. Technically, it is called the topical relevance of a link. What it means is that if I praise you and in my praise I say that you are very helpful, the keyword helpful gets attached to you. If I say you are very artistic, the word artistic gets attached to you.
Now, let’s see how this PageRank algorithm applies to Twitter.
The popularity of a link can be measured by how many times it has been published on Twitter. Moreover, more the followers of the person who tweets the link, more the brownie points for that link.
The keyword context for the link is the entire tweet. A tweet itself is so short, that it the message enveloping the link has high topical relevance.
All in all, the PageRank algorithm when applied to Twitter will give very good results.
Let’s see who is the first one to exploit this 
By the way, for a more in-depth understanding of this subject, check out the trail on Search Engine Optimization. Though the primary focus is SEO, it still has good content on understanding how the search engines work.
Computer Devata
Posted by: Manas in Uncategorized on May 12th, 2009
You go to an office, you want a report from them, you ask them for the report. You can’t get the report yet, you must wait. What could be the reasons?
Not-so-long-ago
- The person who creates the report has gone for tea.
- The file from which data needs to be copied is missing. Someone is trying to locate it.
- The person who needs to sign the report missed his bus and is still on the way to office.
And many more like these. But what’s the single reason that I get today when my work can’t be done by the concerned officials?
- The server is down.
That’s all. I couldn’t collect a medical report from Manipal hospital because their server was down and they couldn’t print the report. My Bangalore Airtel numbers are not getting provisioned because their server is down. Once I stood on Deccan Airways window for a ticket for 3 hours because their server was down.
The life is moving online very fast. However, we are not doing enough to make the online world reliable. The common practise is to just move all the operations and the data online and assume that it will always work.
But the reality is that it doesn’t always work! Wake up guys, build reliability and the speed into your system. Don’t assume they come for free. On the contrary, they cost a lot of money.
Emerging Opportunities in Web Search
Posted by: Manas in Web Search on March 4th, 2009
We are at an interesting time with regards to Web Search. On one hand, the technology (in the hands of Google) has matured, on the other hand, a new window of opportunities is opening up in this space. This presentation explores how the web search will evolve from where it is today!
There is something good in everyone
Posted by: Manas in Development, Misc, social on February 17th, 2009
I recently got to know that in my daughter’s school, all teachers have an interesting annual target. The target is that at the end of the year, every child in the class must receive “Certificate of Excellence” for at least one area. The belief is that every child is good in at least one thing. If the teacher doesn’t find anything good in a child, she isn’t looking hard enough. So, a teacher must find something good in each child and then help the child develop it.
something good in everyone
I was so amused by the thought and compared it with the corporate culture of our times. In fact, I shouldn’t call it culture as there is nothing cultural about it. Anyways, …
So, I was wondering if there be a company in which every manager has a similar performance goal in additional to all the regular ones like meeting deadlines, achieving sales targets etc. Each manager must identify at least one area for each direct reportee where that person can excel. And then through the year, help that person demonstrate his potential in the chosen area.
And this can be completely unrelated to the work for which that person has been hired. If someone is good at painting, there can be an exhibition where he/she can show his/her talent in the whole department or the whole office building. If someone is good at organizing hiking trips, some outbound trip is planned where he/she is incharge. If someone is good at singing, there can be informal singing sessions past working hours.
These are just examples. The skills and the way they are brought out can be different. Even making it part of the performance goals is an example. A company can choose its own method of implementing it. The important thing is the belief that there is something good in everyone.
Won’t that company have a lower attrition rate and higher productivity?
Won’t more and more people want to work in that company making it easier to attract people?
Will you not like to work for such a company? Will you not like to build such a company?
The earlier you start, the easier it will be.
Start now…
Leader’s fall from grace: From Belief To Pride
From the book “One Hundred Years of Solitude”
One night, Army General Aureliano asked Colonel Gerineldo Marquez:
“Tell me something, old friend: why are you fighting?”
“What other reason could there be, old friend?” Colonel Gerineldo Marquez answered. “For the great Liberal Party.”
“You’re lucky you know,” he answered. “As far as I’m concerned, I’ve only just realized that I am fighting out of pride.”
How many times has it happened with you that you took up a project or a challenge because you believed in a cause but ended up just fighting for your pride?
Be it a college election where you start out with a dream to improve faculty-student relations but you end up fighting with the focus that you don’t want to be a loser.
Or you may start a venture now because you believe that your idea can impact the world in a positive way but you end up dragging it along because you don’t want to accept defeat.
Or you may take up an assignment in your company because it will make your product better. But as you struggle with the inertia of other teams, you take it onto your pride to see the assignment through.
Examples are plenty, the net result is one. You start something because you believe in the cause and you just end up fighting out of your pride.
Why? Because as a leader, you struggle with several opposing forces. As an agent of change, you’ll face resistance, offense, and insult. People will come and praise you which will bloat your ego. People will also come and offend you which will hurt your ego. At times, you’ll try to move things and they’ll move, bloating your ego. And at other times, you’ll try to move things and they’ll not move, leaving you frustrated.
When things oscillate between appeasing your ego to hurting your ego, you develop a sense of higher self-esteem as well as a strong urge to protect it at all costs. Well, there you go. You started with the goal of making the world a better place and ended up with a self-centered life.
But fighting for your pride will not take you far. People will not support you. People don’t support leaders. People support a cause, a belief, a dream.
People didn’t support Mahatama Gandhi, they supported movement for independence; they supported method of non-violence. Imagine Gandhi fasting for several days not because he believed atrocities on the poor must be stopped; but because he was up against the district magistrate and wanted to teach him a lesson. Would he have got the support from people they way he did? No. People wouldn’t have even noticed him. Or would’ve laughed at him.
What made the difference for Gandhi? The fact that he was just fighting for the cause and not for his pride.
So, as a leader, stay focused on the cause. If you stop believing in it, just quit and walk away. There is no point in fighting for the pride. You’ll fail anyways…
Your top priority is just another errand for others
Posted by: Manas in Development, Misc on January 29th, 2009
Finishing any sizable work always takes more time than expected, more than estimated, more than it should. Why?
Because for every work that you do, you have external dependencies. There are small bits of things that need to be done by others for you to complete your work. And those small bits of work are as important for the whole work as the big chunks are. Just as a small needle in a sewing machine is as important as the big motor driving the machine.
And those small bits of work need to be done by others. And even though these small things hold the highest priority for you, for others, it may be just a side thing; an errand to be run once in a while or when there is a mood.
And surprisingly, there are just too many of these small bits of work in any sizable amount of work. And unsurprisingly, because of their size (or rather lack of it), these bits are never visible when you make your grand plan. When you make the grand plan, you see big things and budget for them. But small things? Because they are invisible, they won’t be budgeted for. And they’ll take longer to finish because of external dependencies! After all, your highest priority item is just another errand for others
Caught ya!
Anyway, there is no point in holding a magnifying glass and looking for small bits when estimating time. It’s too much of effort and you won’t be able to do it anyways (remember, these are too small to be seen when your mind is occupied with big items).
Just go by the rule of thumb - it takes twice the time, twice the money and will give you half the returns than what you expect.
What’s the correct length for blog posts?
Yesterday, in the course of a chit-chat with Navin, the discussion turned towards blogging and questions like what’s an ideal size of blog posts and how many posts one should write in a day etc surfaced. Since, there were some “insights” in that discussion, I thought I’d summarize them here.
1. Frequency of posts - General principle, the more the better (in terms of visibility). If the same quality of content is written, the blog with higher number of posts gets more visibility and readership. Somehow, the more you produce, the more you get noticed. You are noticed more by the search engines; there are more incidents where your writing can be quoted; there is a higher chance that everybody will find something interesting on your blog. It may also be equated with the more you speak in a meeting, the more ears you get. Of course, there is a limit to everything and you shouldn’t be talking rubbish just to keep high frequency.
2. Length of posts - Short posts are easy to consume but the shelf-life is very short. They are read and forgotten in a very short time span. They cannot go in depth of any subject. Their purpose is more for making people up-to-date or providing some entertainment. However, long posts that go in great depth of a particular subject, last for long. They can be referenced for a long time to come. They are like pieces of art. At the same time, much like great pieces of art, it takes a long time to get recognition for them.
You can also write a series of small to medium size posts on a subject and eventually roll it up in one large, insightful post. Or write a large post on something else altogether. Basically, a healthy mix is important.
3. How much time to spend on writing the title? - For some idealists out there (myself included), manipulating the post title for maximizing attention sounds like dishonesty. But that’s not necessarily the case. Just as a book is judged by it’s cover, a post is judged by the title. It is based on the title that I decide if I want to read a post or not. Title is the essence of a post. So, as long as the title doesn’t promise more than what the post has (check this bait-and-switch ploy by a matrimony site), it’s ok to hand-craft it. In fact, it must be hand-crafted.
What are your insights on effective blogging?
Pushing for “more” - Good side and bad side
Motivated by a post from my friend and my own experiences with work pressure, I was tempted to write something about it.
If you are wondering why you should know about dynamics of pressure in work life, I would assume that you haven’t tasted it yet. I would like to congratulate you for this accomplishment. However, for a lot of poor folks like me, work pressure is an everyday reality which can cause very bad things like early aging and various other disorders.
So, for a lot of people, it’s important to understand pressure.
Usually, people are not so bad that they intentionally pressurize others. Things always start with a push and that push becomes a pressure eventually.
We are living in a time when pushing oneself to be more, to do more, to achieve more is very common. If you can run 5 kms a day, you push yourself to run 8 kms a day. If your team delivers 1000 lines of code everyday, you push them to do 1500 lines of code everyday. If you are selling 50 switches every month, your manager pushes you to sell 70. If your revenue is $20 billion, you push yourself to make it $25 billion.
Nothing wrong here. It’s good. Pushing is a necessary ingredient for the “more” recipe. If your team member is just idling and not starting his work, a slight push may get him started. If your product has high number of bugs, a slight push may motivate people to find innovative ways for reducing bug count. Whatever your sales figures are, a push on the sales people may make them more efficient or creative and get your better sales.
So, pushing is not necessarily bad. If there is room for “more”, it usually gets you more. However, if there is no room for more, or if one is not able to find room for more? What happens then? That’s when push becomes pressure. And that’s a very very bad thing. Because when there is pressure, you don’t get even as much as you were getting originally.
When a team working at the peak is pressurized, the productivity falls. It may still achieve “more” in short term but not without a cost. Development teams that are pushed beyond their limits write crappy software. Creativity goes for a toss. Also, such teams may see more attrition.
Pushing is like a knife. An adept can put it to wonderful use but in the hands of a monkey, it can cause a havoc. So, it’s very very important for us to understand the difference in pushing and pressurizing. Pushing should be a tactic and not a habit. Used as a tactic, it works for you and used as a habit, it works against you.
Unfortunately, neither our education system, nor our corporate training teaches us how to use this tool. We all are part of a network where everyone is pushing one another, either directly or indirectly. Yet, how well do we understand or use it?
What are your insights with push and pressure? When did it work for you and when did it work against you?
RSS