How to build a Technology stack

 It is always exciting to start a new project from scratch and build a new technology stack and see through completion. Building technology stack is very important in any new development project. They play a vital role in successful completion of project, there are some Mind-sets, Strategies and few Criterion that needs to be considered when creating a new stack.  We will try to keep this article as technology independent as possible.

Mind-sets 

Business first  

Always business first, you should create any technology stack based only on requirement and stakeholder needs and not on your love towards particular software or technology or how hot it is or how it increases resume weight-age or your current resource knowledge.  

Even though you consider the current available workforce knowledge while choosing a particular software, you shouldn’t be choosing above requirement need. Any choice must put requirement first and their hindrance later. Once you have validated enough software for one requirement then you can bring in the developers knowledge.  

Kiss 

 As per the saying keep It Simple not Simpler. Don’t do anything that is very complex that needs extra training while choosing a new software. Any software that goes into our technology should have been tried by someone and make sure there are lots of helps online. When you compare two software always take the simplest one which satisfies your stakeholder need. 

Technology stack documents should be created so it can be understood even by non-technical stakeholders, most of the time you need to present the technology stack to non-technical stakeholder and they will be the one who will approving the software selected(budgets). Always present those information how it is beneficial to them not how HOT the software or technology is. 

Define Architectural principle 

   Define architectural principle for your project based on requirements. Keep the principles as much as technology independent as possible. Principle should be crisp and elegant. These principles will help in validating your technology decision that has been taken.  You can ask people to validate themselves before review of technology/design by discussing with other team members based on defined principles. You can define technology dependent principle but that might constraint people from taking decision based on other alternatives. All the principle defined should be clear and crisp. To know How to define architectural principle look here.  You can also refer 101 architectural principles here 

Current development resources 

Final criteria that needs to be considered and it should always be FINAL criteria, the current development resource that we have is very important in case of building a new project out of the box. Choosing technology that has both solved the business requirement and also the knowledge we have, will make the project close to success. Never choose technology only because few of the resources are good at it, business requirement should always precede any other limitations.  

After building technology stack, start recruiting resources, don’t recruit people because you like them, recruit only when they have already gone through the path which your project has to travel so  they will be very asset to the team. Always recruit team players and team has to jell as much possible.  

More people talk to each other, more fun to work together. Create a habit of learning things from people ask them something that you don’t know even if it is small information, be their student on things that they know. You will make it contagious habit that people will ask around a lot and they will leave their insecurity to learn even simple things by asking. Always make it a mantra that everyone cannot know anything but make sure people are also not disturbed all the time  

Strategy

  While choosing any technology make sure you always do a research that the technology/software has already been tried before and also they are successful in releasing their product. You always choose software that’s n-1 of latest version so we have lots of help online and we will know where the quick sand is. Make sure the software you choose has support team that supports the software and the current version. 

When choosing third party software for your project make sure you choose product that you have already tried in your previous product, this is the place where we can consider into account the current development resources prior knowledge, because a software should only be useful to project, not an hindrance. 

Main strategies that needs to be taken care before start building the technology stack is to understand the requirements clearly and in depth (da...!). The strategy I always use is to define three things.  

• Identify functional elements 
• Identify interfaces 
• Find external elements 
• Identify connectors 

Identify functional elements 

Identifying all the functional elements is the key to the success of the projects. What the project is going to serve for the end user should be identified and reviewed with the stake holders and should be documented and approved and this is first step for any architectural document creation. A functional element is a well-defined part of the run time system that has particular responsibilities and exposes well defined interfaces that allow it to be connected to other elements. In simplest form, an element is a software code module. e.g.:- application package, a data store and also a complete system. 

Identify Interfaces  

Identifying how the functional elements talk to each other and how the communication being taken care of the between these elements. An interface is nothing but a well-defined mechanism by which the functions of an element can be accessed by other element. e.g.:- Web API and single Page application talks using REST methodology 

Find External elements 

Before building anything technically, find out the external entities that your application interacts with. Finding all the external elements is the key for the success of the technology stack that is built. We need to find the external elements and find out how to interact with them and how our system will interact with them and vice versa. For example, the database that provides data for the system is an external element and the output that will be submitted to the user in any form is an external element. There may be more than one way our system might get data. There may be many external system that is providing data to our system. 

Describe connectors 

Connectors are the piece of your architecture that link the elements together to allow them to interact. For e.g.: Web API and single page application communicate through the IIS server.  

Concepts of Building technology Stack 

          Here I have used abstraction concepts to build each item in the stack, this seems to be better option because it make sure we don’t miss any of the software that is needed in our system. In this approach we will choose the bottom up approach.  We will start with operating system and will build up to the UI software.  

• Operating system 
• Operating system Abstraction layer 
• Specific application modules 
• Database level 
• Cross cutting concerns  
•  UI Elements 

For the explanatory purpose we will choose a claim process probably a desktop application, where user claims for bonus after he has sold a certain amount of products, bonus amount differs based on products sold. Bonus system is available on mainframe system so you will exposed with a service where you need to post the claim to the mainframe system. We also need to handle offline claims from the system where the user will save claim and post it to certain email id in an excel format.  

  

Operating system 

           This is the place where your client/server software is going to run on. The operating system is an essential component of the system software. Operating system should be chosen based on based on factors such as cost, performance, and security and application usage. 

 Let’s compare Linux vs. windows server, choosing Linux will be cost effective since the Linux OS is free and the servers are less costly when compared with other OS servers. Linux is very flexible where we can do kernel modification according to our need which is not possible in windows. Downside of the Linux is it need a higher level of expertise to keep going in the production environment. Windows  server, on the other hand are easy to configure and maintain. Windows control access is very easy to setup without any special add on which is not possible in Linux. Support in Microsoft is very monolithic because all the products and Microsoft add-ons are supported as one, so you need to contact Microsoft alone but in case of Linux you need to know whom to contact for what needs.  

Note: Make sure you always develop your application with your platform which you are going to support in production too, otherwise we will be facing lots of issues while production release. You can develop Linux based application in windows world by using virtual development environment such as Vagrant or Virtual box. Also consider Microsoft Azure and AWS. 

Operating system Abstraction layer 

The OSAL consists of a set of interfaces (abstract classes) that provide all the required operating system services for the application, including: 

• Tasking services 
• Synchronization services  
• Message queues 
• Communication port 
• Timer service 

When designing your application we might need some need some service that is not provided by the operating system but are readily available in market that satisfies our requirement. For example for asynchronous operation we might need a service that takes care of the need, like message queue and it serves lots of purpose tasking synchronization for A-sync operation and also it gives lot of other features such queue manager.  

Choose these service only when you need a particular operation that cannot be handled by your operating system. Choose technology as light as possible and also it has already been used in many projects. For e.g. you chose IBM-MQ or Rabbit-MQ based on your project, IBM-MQ is heavy but it serves lots of purpose and it has been used widely and IBM supports it and Rabbit MQ is light weight but it is new in market so make sure you select which serves you best. MQ can act as tasking service as well. It also enables us the communication port as well. We might need one for processing our claims submitted through excel files. 

Specific application modules 

This is a huge piece of technology selection that goes into your project. This is where mostly functional and non-functional requirement bonds together. Choose wisely here. Choose technology so that it satisfies all stakeholders, such business and development team. Take care in choosing the specific tools and make sure all the dependencies with other application should also be considered.  Let us consider our example claim process, in which user 

   “User should be able to process offline claims through emails through excel” 

For the above scenario, we need to handle the claim through email and also process excel file, which will be send through the email. So we need a technology stack that suits the excel manipulation and also process and send email. For processing excel we need to have a utility that fits our bill. Microsoft excel can be used but it cannot be installed in server, so it cannot be used to process excel we got through email.  

We can choose ASPOSE which is very much useful for all the office product manipulation. Highly recommended and I have used this in our project many times it is very light weight and it supports both java and .net platforms.  

And for email receiving we need to have SMTP server configured that listens on certain email address and the sending email back with processed result we need to use XSL transformation which is like elixir of formatting email.

We have also have to choose business modules where we need to handle business workflow, business components and business entities. Workflow means the processing the claim and also sending the process details to the respective for the further process. Identifying the business entities and also understanding the process of handling with technology is the key here.  

Database 

When choosing database we need to take care of all the underlying network that will be used in the project, this is the core part of the project where we get all the data we need and also all the data that are processed, saved and retrieved back. We need to consider three thing to consider when choosing the database. 

• Where the data is stored 
• How the data is accessed 
• Needed helpers and utilities
• Service agent used to access the data. 

Where the data is stored 

• Performance vs Consistency
• How much of ACID
• Availability vs Integrity 

In case of ACID compliance database, we need to select the SQL server database. Performance and scalability is your concern and not consistency then go for No-SQL database . If you are going to save documents then choose document database such as Mongo DB or couch database. For our claim example we will be using sql server for acid compliance. We can also choose database based on client available database or the database that is free to use such as MY-SQL. 

How the data is accessed.  

       We need to choose technology in which the data is accessed, retrieved, processed and persisted.    Most common approaches is using Object/ relational mapping(ORM), examples are Entity framework, N-hibernate or use just a database with CRUD models with stored procedures. Choose the technology wisely here. 

For our claim database we will choose the entity framework since we do not want to create a logic for mapping the business entities and database entities because mapping which will be taken care by entity framework and the deployment of the entity framework is also easy where we can choose code first or database/model first approach.

For our example we will be choosing the code first since it gives more control on database version and maintenance is easier. For details about the approach look here 

Service Agents 

When a business component must access/provide data from/to external service, you might need to implement code to manage the semantics of communicating with that particular service. Service agents implement data access components that isolate the varying requirements for calling services from your application, and may provide additional services such as caching, offline support, and basic mapping between the format of the data exposed by the service and the format your application requires. Example for service agents are WCF, Web-API etc. 

Common modules/Cross cutting concerns 

I will not go into details but these are list of common modules you need to be concerned. Choose appropriate technology based on your choice of programming language and your current requirement. It is a blog on it’s own. 

• Authentication  
• Authorization  
• Caching  
• Communication  
• Configuration Management  
• Exception Management  
• Logging and Instrumentation  
• State Management  
• Validation  
• Localization 

We will be leaving the UI layer out of this context.  

Question while choosing technology/Third party software

1. What are the exact installation requirements? 
2. Setup available?  
3. Does it require a machine restart?  
4. What changes are required to the window firewall? 
5. What version of OS are supported?   
6. What are prerequisites? 

Main concept of technology stack building is the mind-set to keep it simple and technology should satisfy only stakeholder’s need. Understand the requirement first completely and  you need to identify the functional elements and external entities first. Once the external entities and functional elements are identified then the use the abstraction layer concepts to identify technology for each layer and finally always ask the questions that are listed in choosing technology/third party section. Happy technology stack building.