Global Sourcing, Out-Sourcing & Offshoring IT software solutions: software evaluation and software selection made simple. avoid failure and common mistakes    

Back to 1st part of Article >>

Prioritizing Needs and Comparing Results     

To effectively match user needs to vendor solutions, users prioritize their needs in a manner that gives greater relevance to criteria that are considered more important.

Weighted averages (WA), are commonly used techniques in selection projects because they are simple to calculate. It only appears to estimate performance reasonably well because it does not indicate which pattern is ideal for an enterprise’s needs, nor does it identify or measure these patterns as a risk inherent to the choice.

Because it uses one number to describe a pattern of criteria, without compensating for any imbalances, a WA falls short of discerning a vendor’s true capabilities.
However, a weighted average composite index (WACI) is a calculation that takes into account the consistency of scores across all of the criteria, in addition to the vendors' weighted average.

A composite index can be used to better understand data that is obscured by the WA. WACI combines the composite index and the WA and measures a vendor’s overall performance against the pattern match.

TEC calls the resultant number the BestMatch Factor™ (BMF) in the evaluation center. BMF is a powerful companion to weighted averages.

Users set their needs using a scale. For example, the scale in the TEC evaluation system is

Critical (10 + a threshold condition for the minimum acceptable rating)
Must Have (10)
Very Important (8)
Important (6)
Nice to Have (4)
Not Important (2)
No Need (0)

The numbers in parentheses represents the numerical equivalent value (that is used in the calculations) of the listed items. Internally, TEC’s evaluation centers will use these numbers for user priorities.

Priorities need to be properly grouped and nested within a hierarchy in order to best determine their weights allowing users to drill-down into the details of each module and submodule.

For example a pie chart (see figure 2) showing the distribution of the current priorities for the selected level in the knowledge tree and a corresponding list of priorities, which are arranged into modules. As needs are set, the pie graph changes to reflect priority settings.

Once all of the user’s priorities have been defined, the WA and BMF are used to illustrate how the selected vendors measure up to the user’s requirements.

Because this data has been collected and analyzed in a knowledge tree, users can pinpoint the results on different functional categories and see how the vendors performed in a particular area. Costing can also be calculated and compared using WA and BMF (see figure 3). Ultimately, these scores allow the user to compare vendor solutions, side by side, accurately, and to avoid the pitfalls of using a spreadsheet.

Graphs of Product Comparison, and Reports

As previously explained, technical and functional criteria are not the only variables that need to be considered when evaluating and selecting software, such as service and support.

Calculating what-if scenarios enable users to determine if the impact of these variables on different vendors. (see fig. 4)

The original results are immediately recalculated, and the vendors strengths and weaknesses are depicted graphically in the Product Comparison graph (figure 5) across the selected level of the knowledge tree.

This graph provides users with a detailed overview of the selected systems’ strengths and weaknesses.

Users can immediately pinpoint key differentiators between the solutions and drill down to investigate where the gaps between the solutions are most prominent and what their sources are.

The Product Differences graph (figure 6) illustrates the gap in performance (WA) between the selected baseline product and others. The selected vendor is displayed and is the Base option.

It acts as the zero line in the graph. The WA of the currently selected level is compared to the WA of the base option on that criterion. The difference is what is displayed on that graph.

This is done for every criterion on the currently selected level in the knowledge tree.
The Sensitivity Analysis graph (figure 7) allows users to compare solutions based on a long-term business vision to verify if a software package will still be an ideal solution based on future needs.
It uses the currently selected criteria as a baseline, and varies the priority for that criterion between 0 and 100 in intervals of 10.

For each priority setting, the overall WA is calculated and displayed for each selected option for the project. Alternatively, the Sensitivity Analysis graph can also be used to determine if a product’s current capabilities can fall within a range of volatility or uncertainty related to certain key business requirements.

One of the most crucial aspects of the software evaluation process is management buy-in.

Given the large expenditure and the impact an enterprise system can have on an organization, it is essential that there is ample support to justify selecting a particular vendor, and to illustrate that the selection process involved rigorous business analysis.

The valuable information that has been compiled during this process is available in a number of reports that can be printed out and delivered to upper management and decision makers.

As a result, the selection team will have quantifiable information to justify a selection.

Additionally, these graphics and reports can be used during vendor demonstrations to separate the marketing pitches from product functionality.

Scripted scenarios can be created based on the prioritized hierarchy of criteria, where vendors will have to demonstrate their products’ functionality in relation to the organization’s needs. Consequently, the selection team can further validate its selection.

Organizations benefit from enterprise software only when they make the right selection. By using accurate and relevant criteria on enterprise software functionality, organizations are better informed of their options.

By avoiding spreadsheet calculations, the selection team can make accurate assessments about how well a vendor can satisfy the organization’s needs, which should ensure the most appropriate enterprise software is selected.

Place Your Ad Here and Benefit!