Saturday, June 21, 2008

Wah!!! What a Customer Service

I want to share my experience with BSNL customer service. Last month I shifted my house to New Thippasandra and had given a letter to transfer my BSNL connection to the new address. The letter was given on 26th of April, 2008. This was given at Indiranagar BSNL office (in 80ft road). Initially they told that the transfer will be done in 3 days. One week passed and still no trace of getting the connection. On 4th May I decided to visit BSNL office and enquire about the status. The person at customer service (in 3rd floor) told that my old connection is yet to be disconnected and right in front of my eyes she was pressing commands to place that order in the system only on 4th May (and that too because I visited. God knows when they would have placed the orders if I hadnt). Then again I visited them couple of days later. This time they told that my old connection is disconnected and now to get a connection at the new address I have to contact the engineer who is dealing with Thippasandra area. I went and met that engineer and told him that I am awaiting for my connection. He noted it down on a piece of paper and told me that it will be done shortly. I also gave him my mobile number to contact and asked him to call me before coming home to give the connection. He noted down my number also in the same paper.

Now the great customer service of BSNL starts. The engineer who took down the orders went on a week long vacation. No one other than him knows that he took down the orders. There is no centralized system to log in customer requests. I visited BSNL again a couple of days later. This time another engineer took down my orders again in a piece of paper but again no follow up. This continued for another week and by that time I had visited BSNL 4 times and each time someone is taking down orders but none is executing. Finally I got fed up and decided to lodge a complaint to Bangalore East Divisional Engineer. I went to his office the next week but the time I went he was not there. So I met up one more officer sitting in the Divisional engineer office's front desk. I explained him my frustration and told him that no one is logging the complaint in a centralized system. He gives great response which is the highlight of their customer service

"NO SIR. THERE IS A CENTRALIZED SYSTEM TO LOG IN. THERE IS A TOLL FREE NUMBER YOU CAN DIAL AND LOGIN YOUR COMPLAINT. THIS NUMBER IS GIVEN IN YOUR TELEPHONE DIRECTORY."

Hearing this I was laughing my guts out. The toll free number can only be dialed if I have a connection. I need not explain beyond this.

(Note: Though I got such a bad response at least this guy was helpful. I explained him the fault in his statement and he immediately called the engineer and ordered him to give the connection. The next day I got it.)

Sunday, June 15, 2008

Operations issues to set right in a startup

It has been 3 years since I left the comforts of a big organization and started working in the startup mode firefighting every day. The learning curve for me in the past three years have been phenomenal both technically and operations wise. Here is my learning on what a startup company should address in terms of operations related issues

1. Lab and network infrastructure - Dont under estimate this. A poor network and lab infrastructure will blow in your face just when you don't want it. In my experience I had come across a unique situation where the servers in the lab were just having a hard disk wipe out just because the AC in the lab was not adequate and temperature in the lab was soaring beyond 35 degrees centigrade and the temperature near the servers was like a hot tandoori oven. Outsource your infrastructure maintenance to people who know it best rather than you fighting it daily.

2. Developers are your primary resources - Value them. Take care that developer's comfort is of utmost importance. Not every startup can provide a cozy office for the developers. But at least see to it its not a cramped space. Giving developers laptops (rather than desktops) and providing them the freedom to sit and work ( a bean bag office environment) in the place they want will save you cost and also make the developers comfortable

3. Power - And a place like Bangalore needs utmost attention in this regard. Make sure you go for a trusted vendor for UPS and also get a consultant to help you in setting up the power connections and proper earthing.

4. Wiring in conference rooms - This is the place I hate to have wires floating around here and there. If 10 people sit in the room and all have their laptops connected to network through wires imagine the mess when someone gets up and walks around. Get a Wifi connection for your conference room. Also get the connection from your projector to power socket go through a sealed cabling. This is another candidate for getting tripped.

5. Have a concierge - Developers need undisturbed time to think and design. Let them not worry about paying their utility bills or getting the weekend movie ticket. Have a concierge setup to help them out. It doesn't cost much for the company to setup a concierge.

6. Keep your hiring standards high - Never compromise on this even if you are hiring for a maintenance / support job. If you have made your first release to market and got into the support mode of it don't think you can afford to lower your recruiting standards to get support staff. After all they are the ones who are going to fix your bugs. Having a bad developer fixing your bugs is asking for disaster. They will ultimately screw up your fundamental design and the entire code will be nothing but a set of patch works.

Friday, June 13, 2008

Solaris Kernel SCTP Issue

We recently we came across an interesting issue with Solaris Kernel SCTP. We observed that if you write a SCTP server program and then initiate thousands of connections from a SCTP client program loop, the server is able to establish the SCTP associations. The point to be noted here is that to simulate thousands of connections from client, the client program loops, creates a socket, binds to the client machine's IP address and a different port number each time. With this setup we were able to create thousands of associations towards the server. But when tested in a live setup where thousands of client machines (each a different machine and hence a different IP address) try to connect to the SCTP server, the solaris kernel sctp goes into a high CPU utilization of near 100%. Interestingly all the clients were bound to the same port number. If we change the port numbers used by the client machines, the CPU utilization @ server comes down to normal.

My hunch here is that kernel SCTP's hashing mechanism to index the hash table of SCTP associations is using only the remote client's port number part to generate the hash value and not the remote port + IP address combo. So if all clients are bound to the same port number, then the hashing algo will generate the same hash value. This will lead to all the client connections colliding to the same hash value thus effectively reducing the search to O(n) or O(logn) based on the collision resolution algo being used.

Refer this for the complaint.
http://forum.java.sun.com/thread.jspa?threadID=5288890