Lessons in Software Development from Alok Srivastava

Editor’s note: This is a guest post on software lessons learned from Alok Srivastava.  Alok is a solution architect at Microsoft with more than 17 years of industry experience.  He specializes in software architecture, large-scale systems, and distributed computing.  Alok has filed for a dozen patents, published several papers, and presents his ideas at many conferences. He has a wealth of experience across large scale projects, database extensibility, multimedia and content management in RDBMS, Web services, portal technologies, and content distribution networks.  In this post, Alok shares his lessons learned …

These are my top lessons learned drawing from my years of software experience.  I am sharing them with you here.

Summary of Lessons
Here is an index of the lessons that follow:

• Lesson 1. Software development is a team sport.
• Lesson 2. More lines-of-code does not mean better software.
• Lesson 3. The Cloud is an inflection point.
• Lesson 4. Scalability, performance and diagnostic ability are better achieved at design time.
• Lesson 5. User experience and user expectation change continuously that is why UI projects are never done.
• Lesson 6. Software maintainability is a key to longer life for any software.
• Lesson 7. Development process should help development produce good quality software, if it comes in your way change it.
• Lesson 8. Take agility with a grain of salt; result –oriented software development is what agility should help you gain.
• Lesson 9. A great software engineer never stops working.
• Lesson 10. Know the keys to writing great software; magic isn’t one of them.

Lesson 1. Software development is a team sport.
Complexity of software systems has grown over past several years. If a software solution has to reach a customer, they need a lot more than just code. They need to install, configure, administer and monitor what they are getting. Developers may need to program with the platform that you are providing. They may also need to bill for usage, measure performance, increase load and add users. At the same time they need to be able to learn, get help, and localize the software for their audience. If the software doesn’t work right, they need to be able to tell you what is not working right. The list goes on and on. Just the software that solves a specific problem can not cover all of this. Developing an industry strength, enterprise class software isn’t something one person can even comprehend. You need a team of folks who are good in their areas of responsibility. This team includes, program managers, development leads and developers, quality team, configuration and install team, management tools team, documentation and release team and of-course evangelists, trainers, support professionals and consultants.
When I started to think about bringing my ideas to life, I could prototype and come with demonstration quite easily without depending on anyone else but turning those ideas to product was a whole different story. I needed a virtual-team that could help me with everything I needed to get the product to price list. Even while working on a startup, a team was needed to get the concept to customers.

Lesson 2. More lines-of-code does not mean better software.
For many years, I have been hearing that the larger lines of code (LOC) somehow indicate that it is better and complex software. I have used that metric in my conversations as well but I have not been convinced that that is actually true. Some of the code that has lived for many years has grown in size as additional functionality was added but one has to understand that back then there was no managed environment and languages such as C# or Java. Operating system may not have helped you much therefore there was a good reason to write all you needed from ground up and that made the code grow fast. Over time even these systems (Windows for example) also have come to realization that they can do better with fewer lines of code. When it comes to managed code environment (like Java and C#) one can do a lot with very few lines of code. In fact, fewer lines of code mean that you are using the facilities that the managed environment provides in the way they were meant to be used. Any time my software team came to me to tell me that they have written a hundred thousand lines of managed code, I had to make them pause and review parts of the code to make sure they were not trying to re-invent the wheel. Given the level of functionality that is available from managed environment, toolkits and operating environment, the LOC measure for code quality complexity should now be outdated and better measures should be developed for the same.

Lesson 3. The cloud is an inflection point.
What we know as cloud and cloud computing started many years ago when Internet started to form. While the most commercially viable applications focused primarily on web site and e-commerce in 1997-1998 early ideas of using Internet as a network that separates users from servers were already taking shape. As usual, it has taken a while for technology to make that scenario viable. Virtualization has also followed the trend to make it possible for cloud to finally form. At the same time, managed environments and Internet based application delivery has eased the fear of losing control if you did not see the hardware running your app underneath your desk. It has taken several computing and IT areas to mature before cloud can be realized. Now that the cloud is here, we are nearing closer to utility computing. If my vision serves me well, we are not too far away from a time when a consumer will be able to create their own way of doing business on the Internet by picking the services that are available and are willing to compete for my business. We are still in learning mode though. Many corporations haven’t yet figured out what the cloud is and what it could mean for their business (or their competitor’s business). Add social interactions on the Internet to the mix and you can see the picture better. Cloud should not be very confusing but when it comes to defining what a cloud is, many people have different perspectives. The way I like to understand cloud is that cloud is a method of service delivery. It defines application/service deployment and access patterns and it is built on services on the Internet (be it platform as a service or software as a service). If cloud is the way we started, the picture would have been simpler and transition much easier. Unfortunately we have many years of legacy dealing with client-server and n-tier applications and it will take time to comfort all weary souls that cloud is all new beast that they need to deal with. Say whatever you like, define it the way you want but the cloud computing is going to have a major impact on our industry and will change the way software business is conducted.

Lesson 4. Scalability, performance and diagnostic ability are better achieved at design time.
This is something I have learned over time and having learned my lessons, the first thing I try to tell anyone writing software is to think hard, design more, analyze as much as you can before the coding begins. When you are first developing application functionality is always on the top of your mind and everything else that software needs takes back seat. I can understand why a software engineer wants to implement functionality first; they get to see the outcome of their hard work. They always say, let’s get it working first and we will deal with the rest of it later. I wish I could give them the budget to build the software from idea to customer and over the life of the software, they will see what I see most of the time. These are the primary reasons why almost all software start ups have to go through a re-architecture cycle before they can have a sustainable software system that works well as their company grows. While I understand that it is not practical to worry about performance and scale when you have to have something in market next month. Many times that is not what pushes the thinking around performance and scale (diagnostic ability is implied throughout our conversation) simply because we don’t want to deal with this. If one could see the overall cost (translate to time spent by many people and re-architecture), they will not ignore these. I would rather extend the design and architecture phase a bit longer in order to save a lot of time and headache later. During design time if your desired parameters for performance scale and diagnostic ability were defined, your design will have to meet those criteria before being accepted. During design time, though experiments and prototyping can help open up the system for large scale and better performance later. If you have thought about it while designing your system (and software) you would know what the impact of scale would be, where your system is likely to become resource constraint and what you would have to do to allow the scale and better the performance. If nothing else you will be able to pre-define the use criteria for your system in order to not fail or get customers in trouble and you will have your requirements for the next release. You may get a chance to put in place low level instrumentation that will help you achieve your performance, scale and diagnostic ability goals during next development cycle even if you did not have time to do it now.

Lesson 5. User experience and user expectation change continuously that is why UI projects are never done.
I have been involved in many UI projects to experience this first hand. The user experience pendulum swings from rich and responsive interface to interface available everywhere. Something that gets the job done or something that keeps people engaged and involved is also a topic of discussion when you are doing a UI project. Sometimes you will also hear productivity argument when it comes to UI. Usability, compliance, localization ability, customizability and personalization are all the terms that get considered when UI project is at hand. First thing that I have learned over time is that no matter how good a UI you think you have there will be some people who will hate that little button. What I am trying to say is that you can never make everyone happy with whatever UI you give them. And that’s not all, people will change over time. If your application is going to be in use for many years, be assured that even those who loved it first may have changed their taste and would not be happy anymore. In short you will never ever be all done with your UI work. So what can you do to succeed in an environment that changes all the time? A few things always help: make it customizable and allow users to personalize (if possible). Define personas and profiles you are targeting, learn from them and have them test your UI before it gets implemented and keep UI work on project every single release so that you get a chance to update the looks as the fashion changes.

Lesson 6. Software maintainability is a key to longer life for any software.
This is a pain most of us working on software that has lived a long life, have faced so many times. Countless hours of productive development time has been spent trying to figure out what a piece of software does, how a system works and what will happen if the code logic was changed. We have all heard of side effects in piece of software that can challenge the best of analytical minds. So why does maintainability get such a low priority in software development. First it is laborious work that has no impact on software features. When your reward system recognizes developers who do magic with their software no matter what they do inside, that’s what they will focus on. If you lucky, you will have that super developer on your staff for more than a few years working on the same software. Unfortunately our industry does not work that way and a career path for a great developer keeps them moving. That brings us back to maintainability of the software. All software is designed with an intention that it will keep selling over time (every tried to figure out why the margins on software sale are so high). If that is the intention, you should keep maintainability on design goal on all your projects. Developing maintainable software is a disciplined approach to software development. It does take time and it is usually difficult to measure but it is as important as some of your features. So what makes software maintainable? Usually having a design/architecture doc is useful but inevitably you will hear that that goes out of sync and cannot be relied upon. Usually what I have seen work the best is using the code itself to keep everything about the code inside it. Who designed it, what it was designed for, what are known limitations, what are the assumptions, did anyone fix a bug there and what did they do, what bug was fixed (a reference to bug tracking system helps) should all be captured in the code or alongside the code. Code comments should be written while code is being written just because they are the most relevant and accurate at that time. Code intent (call it design for the module) should be right there with the module being designed. All developers design before coding, they can easily write it there (provided they are motivated to do that). If a cryptic and hard to read code is being developed, it should need more comments and design details than anything else. The system design (original design) and changes to it should be maintained in sync through code management system (even though it may stress your code management system a bit). There is a lot one can do to save time and money over the life of a code. This is going to become even more relevant with SOA and assembled software. One measure for maintainable software is the time require to have a new developer become productive on an existing software (the shorter the time the better maintained your code is).

Lesson 7. Development process should help development produce good quality software, if it comes in your way change it.
This is my favorite topic of discussion with management as well as developers. I am doing this because that’s what you are supposed to do. The process calls for it and if I do not follow the process, I will not be able to check the code in or something else like that. While I agree that process and guidelines are necessary and important but lets’ not forget the purpose for those. Software isn’t a bi-product of processes and guidelines, software is the product. Process should make it easier to develop software, ensure that all requirements are being met. This is why processes should be reviewed periodically with an intention to change and fix it as necessary. This is one of the many things that I would listen to developers and first level managers for. They should be trained to recognize when a process is not working for them. They should be able to identify shortcomings and someone responsible should review the processes and change it to make sure that the intentions behind those processes are still being taken care of without overburdening developers. Automation helps as well. Simpler, meaningful and reasonable processes and guidelines are likely to aid developers in their daily job as well as help get the best product out and everything should be done to get to the state where developers start to feel that guidelines are processes are their friends and they have a good say in shaping these. Process agility is a key to the success of a development team. It should also be recognized that the same processes and practices may not work well with all the teams. The experience level of developers and the nature of project should help shape processes necessary. Cost of changing process is far less than extra time your developers spend getting frustrated because of outdated and meaningless processes.

Lesson 8. Take agility with a grain of salt; result –oriented software development is what agility should help you gain.
Agile development methodology; if you are not following it, you are not current. It has become a fashionable new term for management to say – yeah we follow agile methodology in our development. While it sounds cool to say you are agile, stop and take some time to see what are you agile about. Agility isn’t just following Scrum; it is a way to become result-oriented software development team. Lots of time I have seen that software development teams are either forced into following agile methodology or they opt into it because everyone else is doing it. They usually know what to do but they don’t know what to expect. What is also not clear to most is that not all parts of software development benefit from agile methods; The truth is that agile methodology can hurt the overall development if it is applied without proper thought. For example the kind of agility needed for an online SaaS software development shop is quite different from what is needed for traditional development products that get shipped to customer. The right participation in agile method is also key. Program management, quality team, deployment and even documentation team should follow agile method and synchronize with other teams at appropriate time. Architecture, design and requirement gathering is usually not amenable to agile methods (at least the same kind of agile methods that something like Scrum proposes). They can be done in progressive detail as a complete phase before development kicks in. Through my experience on reviewing many projects, I have seen that some of the projects took a lot longer to complete even though they followed the methodology as well as they knew how to follow. Agility is a great achievement for any project team but they have to figure out how they need to adopt it, what would be the impact, what are the expected outcomes and which activities should be pre-conditions or post activities. A lot needs to be considered including involvement of all teams in coming up with agile process before actually executing a project with it. The price for making a mistake could be high and sometimes benefits are never realized. The right kind of agility is what helps reach the ultimate goal- result-oriented software development where everyone involved had a clear insight into how the progress is being made, they can course correct as necessary without having to take a blind approach and just go on faith. While one should take great care and do analysis before adopting agile methodologies, not adopting one can do more harm than good.

Lesson 9. A great software engineer never stops working.
I bet you are wondering about this. I thought of picking this topic after hearing (and sometimes practicing) that many teams wanted their developers to log their time to measure productivity and I tried to see how one can measure this. The only thing you can measure is the time you spend typing in from of computer screen but is that really the place where software is created? Sometimes yes but not always. Developing software at its core is a problem solving exercise and when you have a problem, you think about it everywhere. While watching TV, playing soccer, driving or (for some very dedicated individuals) while sleeping. Now how do you measure that as part of your work? The fact is that when presented with real challenging issues, a software engineer will devote all their time until they solve the problem or find someone to help with that. It is just in their nature, their entire engineering training and required educational background shapes them that way. So the real reason I brought this up is to make you re-consider asking your software engineers to log hours to declare how productive they have been. Just use their accomplishment, commitments and goals to measure them with. Forcing software engineers to declare that they work 40 hours a week or more will get you exactly that answer and you know as much as your engineer that there is no real productive measure in knowing that.

Lesson 10. Know the keys to writing great software; magic isn’t one of them.
No matter what development methodology you decide to follow for your software development there are some fundamental things that you should seriously consider accomplishing in order to get quality software to your customers. Keep in mind what I am writing here applies to packaged software development (one can modify these steps for SaaS model but I won’t be covering that in this article):

• Understand requirements and use-cases: This helps your team understand what they are up against, what assumptions they can make and how they can proceed with their activities. Like every good reviewer, challenge the requirements and question it’s validity. Even though your program manager may be presenting the requirements to you, they need to be able to confidently explain and defend each and every one of those. A requirement may be coming from imagination (new markets), customer demands, customer feedback, steering groups, industry changes, competition and so on. It is a good idea to know what they are.
• Assess the needs: Next you should assess the needs for what major software blocks are going to be needed and how they could fit together. In the process of doing this find out what others within your company may already have done (you can simply use their code) or if there are some know publications (make sure you have rights to use it) that address what you are trying to do.
• Gather what you can find and identify the gaps: Gather the software/modules you can get and identify where you will need to develop new software. In the process also identify risk area (areas that you don’t know much about).
• Prototype: There are several reasons for doing this. First you can eliminate or highlight situations that you thought were risky. Get a sense of is the problem can be solved or a different approach is needed and if there are multiple ways to solve a problem which path will be the best to take. You are also getting insight into what it will take to develop software and that will help you plan and estimate better. This is the first milestone where course correction in terms of planning and schedule should be made. You would also find out if you have to license/acquire technology or if your endeavor is infeasible and should be abandoned. If you are able to carry on, this is your first chance to demo something and improve your team’s credibility.
• Design and Architecture: By now you have a great idea about elements needed to build the software. Go ahead and build it … on paper. Write down design, complete architecture, deployment scenarios, test scenarios and also demo scenarios. You know what you need to build and the parts involved, you should be able to accomplish design to a fair amount. If is ok now to become agile and identify isolated areas that have defined interfaces and behaviors but can be designed in detail later. Getting on the paper makes everyone know the system better, question and challenge assumption, incorporate performance and scale parameters, address maintainability and so on. This phase does take time and the whole team is involved but by the end, your team would know the order in which the development will take place, how will systems communicate, how they will fail and recover, are you getting software from inside or acquiring from elsewhere and other details. All that remains is to implement.
• Code and document: the previous exercise will give you enough confidence if not all details and a good plan with reliable schedule to complete this phase faster than ever before and perhaps with fewer high impact mistakes. This phase could get a lot smaller and pain-free if all the activities are being taken up accurately. This is great phase to follow full agile method for so you can show progress as you go along finish design details for you was an isolated module and engage documentation and quality teams to start their work. Installation and release teams can get started right here and start to work with shell of an application while features are being put in. Program managers can start to verify use-cases and at some point engage close customers for feedback. You can track to your schedule and see if you need to adjust. By the time coding is done, very little is now remaining to complete the project and hand it over to quality team for final test and sign off, after this phase is completed.
• Verify use-case, get user feedback, and fix bugs (no new features): Now the software is ready for final phase of testing user feedback, demos, and bug fixes. If everything else has gone as planned before this phase, one should not be surprised if not too many bugs are discovered in this phase and many of the really bad issues should not show up.

What I am describing here is not a methodology for software development rather than distinct activities that should be performed. If you are following agile methods then you can imagine some of these activities will be mixed in as necessary for every iteration. I would strongly recommend not to short circuit design and analysis phase and force that in an agile method since that has a huge chance of making all you agile efforts ineffective or even harmful. I have seen one too many projects suffer through this painful exercise and in the end they did not benefit at all from agile method (although not measured but it may have cost them more due to course correction in between the project). These mistakes will force a lot of back and forth during iterations and will cause developers to wait for corrections to be made or will force them to go back to previous iteration for re-design. Requirement analysis and design phase will help eliminate that and will help better plan and schedule for agile methods to produce desired results.

You Might Also Like

Top 20 Best Software Engineering Books of All Time
Architectural Styles in Software Engineering
Lessons in Software Development from Mike de Libero